In this post I will be detailing how I created a custom hook in React that abstracts the complexities of dealing with fetching paginated data for an infinite-scroll experience.
You'll likely be familiar with the typical infinite scroll user experience that's often seen in mobile applications. We'll begin by listing out the requirements for the eventual user experience:
Despite the user-experience being relatively simple, dealing will all of the states for fetching paginated data is actually pretty difficult. Let's take a look at a few of the states and methods needed to implement the user experience:
In an application that has many instances of paginated data, it makes sense to share the logic for managing and keep track of the state and to provide methods for managing paginated data. However, the utility can't make any assumptions the actual data itself or where it's fetched from otherwise it would be difficult to be shared across many types of data. So, on top of the states above, the utility needs to have the following traits:
As you can see there's a lot of internal complexity for this kind of utility but the external API is flexible, simple and reusable. Let's get in to the code.
I'll be building the paginated utility with React Hooks, and I'll assume that you're familiar with them in the rest of this post. If you're not, I highly recommend an afternoon with coffee and the official React documentation for hooks which do a much better job of explaining hooks than I can.
I typically start with the external API, which helps formulate a blueprint of what the internals requirements will look like. Let's sketch something out:
const {
ids,
loadingInitial,
loadingNextPage,
refreshing,
loaded,
errored,
endReached,
fetchInitial,
fetchNextPage,
refresh
} = usePaginatedFetcher(
(from, size) => () => fetchSomePaginatedData(from, size);
)
So I've sketched out the initial list of things that an external component using the paginated hook will need to use to implement the user experience described above. Here's the Typescript definitions:
{
// Keeps a track of the ids returned from the fetches
ids: string[];
// Whether the data is loading for the first time
loadingInitial: boolean;
// Whether there's already some data and the next page is loading
loadingNextPage: boolean;
// Whether the user has issues a pull-to-refresh
refreshing: boolean;
// Whether any data has fetched, ever
loaded: boolean;
// Whether a fetch has errored
errored: boolean;
// Whether there are no more pages to fetch
endReached: boolean;
// Fetches the initial page, updates the loadingInitial state
fetchInitial: () => Promise<...>;
// Fetches the next page, updates the loadingNextPage state
fetchNextPage: () => Promise<...>;
// Refreshes the data, updates the refreshing state
refresh: () => Promise<...>;
}
The reason that
fetchInitial,fetchNextPageandrefreshdo not have any arguments is because these functions are inferred from the return type of the passed fetcher function. More on the specifics of how this is achieved later in the post.
The last thing to map out is what the fetcher function looks like. The usePaginatedFetcher hook takes in a function as the first argument that is responsible for actually fetching the data from somewhere. This function receives from and size as the first two arguments (which represent which index of records the back-end should start returning, and how many records to return respectively). The fetcher is then expected to return another function that performs the fetching.
The reason the fetcher is structured as a higher-order function is because the hook is responsible for managing the state of from and size, but there could be any number additional arguments required to perform the fetching of the data, like ids, filters, searches, sorts, etc... and the hook shouldn't make any assumptions or restrict the flexibility of the data fetching itself. Here's the type definition of the fetcher function:
type Fetcher = Fn extends (from: number, size: number) => (...args: any[]) => Promise<Paginated<any>>
...args: any[]could contain things like filtering, searching, etc that would later be used by the fetching function itself
Where Paginated is an interface defined as:
interface Paginated<T> {
count: number;
results: T[];
}
Contradictory to what I stated above - We are assuming that the API returns a response in the
Paginatedstructure. Of course, your app will likely be different to this so feel free to amend as necessary, or refactor the hook to be even more generic.
So now that the external API is defined, let's move on to the internals.
First things first, let's set up the necessary useState hooks and derived state to manage the internal (and some exposed) states:
type Fetcher = (
from: number,
size: number
) => (...args: any[]) => Promise<Paginated<any>>;
const usePaginatedFetcher = <Fn extends Fetcher>(fetch: Fn) => {
const [ids, setIds] = useState<string[]>([]);
const [count, setCount] = useState(0);
const [loaded, setLoaded] = useState(false);
const [errored, setErrored] = useState(false);
const [loadingInitial, setLoadingInitial] = useState(false);
const [refreshing, setRefreshing] = useState(false);
const [loadingNextPage, setLoadingNextPage] = useState(false);
const endReached = ids.length >= count;
return {
ids,
loaded,
errored,
loadingInitial,
refreshing,
loadingNextPage,
endReached,
};
};
Though we haven't tied any logic to these states yet, but by defining them, we've provided 7 out of 10 of our external APIs!
And fetching data goes as follows:
const fetchData = useCallback(
(initial: boolean) =>
async (...args: any[]) => {
const from = initial ? 0 : ids.length;
try {
const response = await fetch(from, size)(...args);
const newIds = response.results.map((result) => result[idKeyName]);
setIds(initial || newIds.length === 0 ? newIds : [...ids, ...newIds]);
setCount(response.count);
setLoaded(true);
return response;
} catch (err) {
setErrored(true);
}
},
[fetch, ids.length, idKeyName]
);
There's a couple things to note here. The first is that the fetchData function is a higher-order function that takes as it's first argument initial: boolean. This boolean is required because the hook exposes methods fetchInitial and refresh, both of which fetch data and replaces the existing list. The initial boolean is used within the setIds call to determine whether to append the results to the existing list (in the case of infinite scroll) or replace the results in the list.
The second thing to note is that there's a variable I haven't introduced yet, idKeyName. The hook isn't allowed to know about the structure of the returned data, yet it's expected to keep track of the list of results. These two things are contradictory of course, so there needs to be some way for the hook to know how to uniquely identify a particular result. To achieve this, a idKeyName is passed in as a new argument to the hook and is used to map a unique identifier per record. The list of these identifiers are managed in the state of the hook and passed back out so that the component can map over them and render the results:
const usePaginatedFetcher = <Fn extends Fetcher>(fetch: Fn, idKeyName: string) => ...
Now a reusable fetchData higher-order function has been defined, it's simple to create some fetchers tied to individual states and logic:
const fetchInitial = useCallback(
async (...args: any[]) => {
if (!loadingInitial) {
setLoaded(false);
setLoadingInitial(true);
try {
return await fetchData(true)(...args);
} finally {
setLoadingInitial(false);
}
}
},
[fetchData, loadingInitial]
) as Return;
const refresh = useCallback(
async (...args: any[]) => {
if (!refreshing) {
setRefreshing(true);
try {
return await fetchData(true)(...args);
} finally {
setRefreshing(false);
}
}
},
[fetchData, refreshing]
) as Return;
const fetchNextPage = useCallback(
async (...args: any[]) => {
if (!loadingNextPage && !endReached) {
setLoadingNextPage(true);
try {
return await fetchData(false)(...args);
} finally {
setLoadingNextPage(false);
}
}
},
[fetchData, loadingNextPage, endReached]
) as Return;
Where did as Return come from? Well since these methods are called externally, it's important that they represent the shape of the fetcher function omitting the higher-order bit (where the from and size arguments are injected) that the hook calls with the state of the from and size properties. In fact, Return should be the return type of the fetcher function argument explained above. Let's drill it down.
The fetcher function type looks like this:
type Fetcher = (
from: number,
size: number
) => (...args: any[]) => Promise<Paginated<any>>;
However, we want the methods exposed by the hook to look something like this:
type FetchNextPage = (...args: any[]) => Promise<Paginated<any>>;
With the arguments inferred of course, such that if the fetcher function requires any additional arguments such as ids, search, filter sort etc, the callee of the fetcher knows about it. TypeScript comes with a handy type utility called ReturnType that we can use to our advantage:
export const usePaginatedFetcher = <Fn extends Fetcher>(
fetch: Fn,
idKeyName: string
) => {
type Return = ReturnType<typeof fetch>;
...
}
Pretty neat!
Hopefully you'll get a sense for how all the bits of this hook fit together, but here's the entire code for the custom paginated data hook!
type Fetcher = (
from: number,
size: number
) => (...args: any[]) => Promise<Paginated<any>>;
export const usePaginatedFetcher = <Fn extends Fetcher>(
fetch: Fn,
idKeyName: string,
size = 30
) => {
type Return = ReturnType<typeof fetch>;
const [ids, setIds] = useState<string[]>([]);
const [count, setCount] = useState(0);
const [loaded, setLoaded] = useState(false);
const [errored, setErrored] = useState(false);
const [loadingInitial, setLoadingInitial] = useState(false);
const [refreshing, setRefreshing] = useState(false);
const [loadingNextPage, setLoadingNextPage] = useState(false);
const endReached = ids.length >= count;
const fetchData = useCallback(
(initial: boolean) =>
async (...args: any[]) => {
const from = initial ? 0 : ids.length;
try {
const response = await fetch(from, size)(...args);
const newIds = response.results.map((result) => result[idKeyName]);
setIds(initial || newIds.length === 0 ? newIds : [...ids, ...newIds]);
setCount(response.count);
setLoaded(true);
return response;
} catch (err) {
setErrored(true);
}
},
[fetch, ids.length, idKeyName]
);
const fetchInitial = useCallback(
async (...args: any[]) => {
if (!loadingInitial) {
setLoaded(false);
setLoadingInitial(true);
try {
return await fetchData(true)(...args);
} finally {
setLoadingInitial(false);
}
}
},
[fetchData, loadingInitial]
) as Return;
const refresh = useCallback(
async (...args: any[]) => {
if (!refreshing) {
setRefreshing(true);
try {
return await fetchData(true)(...args);
} finally {
setRefreshing(false);
}
}
},
[fetchData, refreshing]
) as Return;
const fetchNextPage = useCallback(
async (...args: any[]) => {
if (!loadingNextPage && !endReached) {
setLoadingNextPage(true);
try {
return await fetchData(false)(...args);
} finally {
setLoadingNextPage(false);
}
}
},
[fetchData, loadingNextPage, endReached]
) as Return;
return {
ids,
loadingInitial,
loadingNextPage,
refreshing,
loaded,
errored,
endReached,
fetchInitial,
fetchNextPage,
refresh,
};
};
This paginated data hook isn't perfect. For one, it relies on the response of the data from the back-end being consistent, and makes some assumptions based on it, and it cannot be pre-loaded with a list of ids, so the first page will always be fetched, even if there's already some cached state from previous fetches lying around in a global store. However hopefully this gives you an idea of how React Hooks can be used to create powerful and useful abstractions for what's normally a pretty difficult state management problem to share across many components easily.