There’s a problem I’ve faced which I’m sure is familiar to most people: you’re heading home from the grocery store, you pull into the driveway and suddenly remember your partner’s text from this morning saying we’re out of garlic powder. This is the problem I wanted to solve with pantry.
Planning:
I spent about 6 months daydreaming about this app before I actually sat down and took a swing at it so I already knew a bulk of the features I wanted from the app. When creating a project I try to find a problem → reduce it down to it’s most simple wording → write code to solve that problem.
So my complaint was I keept missing things at the grocery store, that’s pretty easy to understand but that’s not the problem. Why am I missing things at the grocery store? The problem is there are too many lists in many places, the solution is one list in every place.
Finding Features:
Goal: a single list on multiple devices with multiple contributors.
Thinking about this goal in the context of trying to extract the unique features; users, authentication, authorization are a given necessity, what is needed to solve the problem? At this point I had a foggy idea of what I wanted the app to look like and I felt like it would be easier to discover the feature requirements by taking a UI first approach. By planning out how I’d like to interact with the app I can nail down what the app needs to do.
I opened Figma and after about 100 dribbble posts and several restarts later I had a sheet of elements, a mock-up of every page necessary for the app to function and a decent starting point for extracting objects.
Personally I don’t think it’s necessary to fully flesh out the design at this point, I start to design the ui at this point because it helps me figure out what features I need to solve the problem which gives me a set of clear goals I can break down into smaller tasks, avoiding scope creep.
Extracting Objects:
At this point I have the front end roughly designed and I switch focus to the back end. I’ve laid out the data I want and at this point I try to extract objects from the requirements.
What I extracted:
- User: should have a name, email and password, friends, and associated lists.
- List: has a title, associated items and many users through contributions.
- Item: has a name, a note and a state (acquired or not). Belongs to a single list.
- Contribution: model that associates a user to a list, belongs to one user and one list.
- List Invite: represents a user’s request for another user to join a list, should have a method to ‘accept’ the invite and create a contribution record.
- Friendship: represents a relationship between two users, two reciprocal records are necessary to properly represent a bi-directional friendship.
- Friend Request: represents a user’s request to create a friendship. should have methods for accepting/declining and triggering the desired side effects.
First Code:
At this point I start building the rails backend api by using the api flag with rails new. The api flag does several things like making; generators skip creating views, the ApplicationController inherits from ActionController::API leaving out functionalities used by browser applications, and starts you off with significantly less middleware.
I start with the models and relationships, then validations, then routes, then serialization, and then authorization. Experimenting in the console is handy when working on the models. I try to avoid scaffolding when possible though I do utilize rails generators.
Validations:
When I’ve got everything hooked up the right way I move on to validations. I find most validations are pretty straightforward while building the models and I make notes in the files while I’m working on the relationships. The app has custom validators for Contributions, FriendRequests, Friendships, Items and ListInvites. That’s too much to paste here so I’ll just go over the FriendRequest validator.
So lets consider a friend request, what do we need to avoid happening?
- We want to make sure there isn’t already a friend request between the requesting user and requested user.
- We need to make sure the two users aren’t already friends.
By using custom validators we can write logic to check for complex conditions and return custom error messages.
Lets run through this code step by step:
- store the requesting user and requested user in variables for easier access.
- First we make a query for a record with the same requesting and requested users, adding an error to the unsaved friend request instance with the message “request exists” if a duplicate is found.
- With the record’s uniqueness verified all that’s left is to check that the two users are not already friends and add an error if so.
Pretty straightforward no? just copy the same thing for the models that need it and you’re good to move on to routing.
Routing
Now we’re getting into the fun part. I used RESTful routing and nested resources in the design of this api.
At the top of the code we have the following explicitly declared authentication routes. These routes direct post and delete requests to ‘/login’ and ‘/logout’ to the create and destroy actions of the auth controller.
After that we have the User resource which we will use to perform CRUD actions on Users as well as access CRUD for Friendships, Lists, FriendRequests, and ListInvites.
For instance, if a User wants to see their sent and pending FriendRequests, they would make a GET request to ‘users/{USER ID}/friendrequests’. If they wanted to accept a FriendRequest the could make a POST request to ‘users/{USER ID}/friendrequests/{REQUEST ID}/accept’.
Resource nesting mimics the way you would do actions in the code or the rails console for example:
is the equivalent to:
GET ‘/users/3/friendrequests
Accessing features and data in a uniform way project-wide that conforms to the root structure of the app’s data simplifies debugging when problems arise and makes making changes easier later on.
I build the routes and controller actions at the same time, adding the route and then creating the action for that route. I started out working on the routes using postman but at the time I had 8gb of ram and postman kept crashing with vscode, the rails app and firefox running at the same time so I started using the vscode REST Client extension by Huachao Mao. Switching to the extension freed up a lot of ram and greatly reduced the wear on my alt and tab keys.
Separation Of Concerns
It’s important to keep controllers lean so I try to push as much of the logic as is appropriate into class methods on the models involved, for example hitting the accept action from the FriendRequests controller should do or trigger several things:
- Create reciprocal Friendships between both users of the request.
- Get rid of the now useless FriendRequest.
- Announce that the friendship has been formed
Let’s look at the code:
We can see this function accept finds a friend_request and a user record using request params, then it checks if the current user is the requested friend and if so, calls the accept method on the friend_request, locates the new friendship and renders it as json.
So what does the accept method look like?
here we see the method accesses the model’s requestor and adds the requested user to their friends pretty simple right? You might be thinking ‘but don’t we need to create a reciprocal record for the friendship to work both ways?’ and you’d be absolutely correct which is why we now need to take a look at some code from the Friendship model.
At the top we see two ActiveRecord callbacks after_create and after_destroy, ActiveRecord callbacks are triggered by ActiveRecord model life cycle events, for instance: just after a friendship model is created rails runs the reciprocate_friendship method which creates the necessary reciprocal record if one does not already exist. Similar to the record reciprocation on creation, it is important to reciprocate the deletion of both records which is just what the remove_reciprocal method does after a record is deleted.
I could have put that logic into the accept action in the controller which would technically work but would leave me with a single action longer than what I ended up with as a finished controller. (spaghetti code) Instead I’ve broken the code up into smaller chunks located in contexts that make sense. The controller action tells the friend request to accept it’s self, and it handles things from there, the FriendRequest doesn’t create the reciprocal friendship records, it creates one friendship record and cleans up after it’s self and lets the Friendship model handle the rest of the work.
Authentication:
For authentication I utilized rails’ built in password and authentication functionality has_secure_password. This allows me to easily authenticate a user’s credentials with minimal setup.
Front End
Now that the back end was basically sorted I moved on to the front end. I anticipated at least 2 more phases: stitching together the front & back end and deployment. To build the front end I was using react and I knew I wanted to use react-styled-components as well as redux and redux-thunk for managing data and handling api requests.
Structure:
The overall file structure of this app went through several iterations as the app grew the src directory is broken up into feature/component folders including common components like forms and static pages like the login and account information screen.
Style:
This project started with react-styled-components, however I found that my ‘elements’ files were quickly getting overfilled and hard to navigate and the imports were complicated and took up a significant chunk of the top of my files. This is a problem I was waist deep in before it occurred to me so the current state of the styling is some things are styled with plain Sass and some things are still react-styled-components.
Though I don’t really like react-styled-components I cannot deny it’s handiness in certain situations, for instance, you can abstract an element that accepts props and use that element in multiple different contexts with multiple states.
Responsiveness:
the frontend is not so much “responsive” but instead mobile first as it was built with a Pixel 4’s screen in mind and anything bigger gets the same ui centered horizontally on the screen. I have plans for a IU redesign and i will write another post when that happens.
Data Management:
For managing and distributing data throughout the app I used redux and redux-thunk. My redux store was constructed from 7 reducers, each handling the data of a specific entity:
Each reducer has an equivalent set of actions defined within action files in ‘/src/actions’. These action files contain both the action creators dispatched to the root reducer and the thunks used to update the store asynchronously. For the most part the action creators are for creating, updating or destroying the the resources, however, some resources such as list members and items also have ‘reset’ action creators. This is because these slices of the store are replaced in full every time a user accesses a different list. We must clear these parts of the store to prevent old and unassociated data from being displayed to the user.
Data Acquisition:
The front end is relying on a separate server to provide the data which means we’ll need to request data when we start to build a page and then slot the data in when we get it from the api. To do this we use thunks to asynchronously request data and dispatch actions using the responses.
This example sends a request to the api to toggle the ‘acquired’ state of an item, if the response is ok it dispatches a toggle action to update the redux state. Alternatively, if the response is unauthorized a logout action is dispatched, wiping all local data, triggering a redirect to the login page.
These thunks are imported into components where functions dispatching them are declared and mapped to props.
Then they are used like so:
Thunks are also used to load data this Thunk makes a call to the api for all of the items of a list, if the response is ok it dispatches a load action with the returned data. If the the response is unauthorized it logs out, if I’ts just bad it displays an alert to try again later.
Now each item card has a button that can trigger an api call to toggle their state!
This Thunk needs to be run when we go to a list page to load the list’s items. To do this you can use the useEffect hook with an empty dependency array like so to trigger on mount:
If we remember earlier, list items are a slice of the store that needs to be reset between different lists, to do this we can use a useEffect ‘clean up’ function, or a function that runs when the component is unmounted, we can do that by writing a function that is returned by useEffect like so:
Now when this component is unmounted clear() will run which dispatches the reset action to the list items slice of the store preventing items from another list being displayed while waiting for the load Thunk to dispatch the response from the server.
API Communication:
When I started working on how the front end would request data from the api I hadn’t fully worked out the serialization of the data or the routes so I was jumping back and forth between the front and backend and tweaking things. So rather than writing out repetitive fetch functions in my Thunks I thought the best practice would be to create an api wrapper.
I spent a big chunk of the total development time redoing this several times until it’s current iteration which I will admit I’m sure I can squeeze a little more abstraction out of for slightly less repetition but I’m building a grocery list app and not the perfect api wrapper so I’m done for now.
I’m going to skip over security for the moment because that has it’s own section later
The Api wrapper consists of a set of functions, that provide a framework for creating requests to the api through a function called baseRequest which accepts an endpoint, a request method (get, post, patch, etc), a boolean for whether a request requires authorization and a request body if needed.
baseRequest is then used in functions stored in resource specific obejects like so:
The toggle function accepts a list and item id and returns a baseRequest with the constructed endpoint to acquire the item, specifying that ‘PATCH’ is the method to be used and that authentication is required for this request.
All of the functions for a given resource are exported from the resource’s file as an object, these objects of resource specific functions are then imported into a central index file and exported as one combined api object for semantic usage of the wrapper:
baseRequest separates the code that makes the request from the code that processes the result. Therefore thunks that call the api wrapper have a simple interface to handle the request and don’t have to worry about request or endpoints.
Authentication and Tutorial Hell:
In the end though I was intending to make a RESTful api and I feel like this breaks that I went with the http-only and csrf token cookies. The strategy uses a session cookie an http-only cookie to store the user id and a csrf token cookie.
Every project has it’s quirks and bugs, that’s normal but whether it’s code or strategy or planning I feel like there’s always one thing that’s just messed up or wrong. If you catch it quick you can fix it and move along, if you don’t then you can waste a lot of time. I didn’t touch authentication until everything was pretty much in it’s final state, I figured it would be simple to implement something that’s so important there must be gems that handle it. For the most part I was correct; if I wanted to use cookies, well that’s built into rails, and If I wanted to use a JWT there are gems for that. This was the cornerstone of my wasted time. I switched back and forth from JWT to an http-only cookie + a csrf token cookie solution several times based on what the last blog post I read was and I remain philosophically conflicted on this issue still. Do cookies and sessions violate RESTful stateless requirements? Isn’t local storage open to csrf and xss?
Here we have a function authOptions that returns properly formatted request headers for a request authorized with a the value of the CSRF-TOKEN cookie in the X-CSRF-Token header.
When we make requests to the api rails will recognize the header and it will be available for us to verify. We can have rails check each request for a valid csrf token by using protect_from_forgery like so:
To create a new csrf token and set it as a cookie we create a function and call it in a before_action like so
Here we create a new csrf-token cookie with the value of form_authenticity_token, If the environment is development no domain is set on the cookie, if it is production it is set to my domain name.
We can write a similar set of functions for checking if a user is logged in:
Here we have a function bake_cookies that accepts a user id and sets it as the value of an http-only cookie expiring in 1 day, setting the domain if in production. Then we have the current user function which uses memoization to return the logged in user or to look it up from the id cookie. Then we have logged_in? which is a simple way to check for the presence of a logged in user. Finally we have authorized, which checks if a request is from a logged in user and responds with a json error message and an unauthorized status if not.
We can check every request for authorization by adding one line to the application_controller:
However some actions cannot require a user to be logged in: logging in and creating a user, to skip the auth check on those actions you can add a line like so in the controllers:
Deployment:
After finally settling on cookies I moved on to deploying the app, I had previous experience with Netlify so it was a no-brainer to host the front end. Similarly Heroku seems to be the industry standard for rails apps so I went with them. I won’t go over the specifics of how to deploy on either of these platforms, their documentation is more than enough and both have very nice command-line interfaces. What I will say is that environment variables in react on Netlify can be slightly tricky so don’t get discouraged if it doesn’t work at first.
I stored several things in environment variables: The base url used by the api wrapper in the frontend and the origins host for the cors initializer in the rails api.
After a bit of tweaking The app was live and functional!
Lessons:
The most important lesson I learned was that though the flexibility to acknowledge that something isn’t going to work how you planned is important and can save you time, you shouldn’t change direction for every blog post you read. If I could go back in time I’d tell myself to pick jwt and be done with it. Switching back and forth not only consumed time purely from a mechanical perspective but I began to resent the project for a period during the development and that sapped my drive to complete it without a doubt protracting my work by at least a week or two. On the other hand I learned some new git techniques, mainly merging a single file from another branch.
I feel the main lesson I learn with every project is that I didn’t plan enough. Not that my planning was bad, it was better than my last project but my next will be better. I’ll plan for things in my next project that I didn’t even consider in my last app’s plans. My next project will more smoothly go from idea to deployment, and at the end of it I’ll say I didn’t plan enough and that’s good because that’s growth
The one area where I feel I didn’t try hard enough was testing. When I started this project I wanted to do T.D.D. and I even started out using it, but after writing and replacing several sets of tests I abandoned them. I intend to write tests for this project in the future, I also intend to try harder to stick to test driven development in the future. I think some of the problems I ran into could’ve been avoided by writing tests first and having an explicit deliverable before writing code.
Thanks for reading!
If you enjoyed my writing feel free to check out my website and connect with me on LinkedIn
