Get all posts from WordPress REST API

As you may know, WP REST API has a limit of 100 records per call. This is what it says in the API Docs:
cap
Now this may be enough for some people, but what if you need to get ALL of them. Here are two solutions that I came up with.

1. Using PHP

We’re using PHP’s multi cURL. This means that we are doing multiple REST calls, but doing them simultaneously. Meaning, the amount of time to do the requests – is not dependent on the number of calls you make (because they’re asynchronous).
So let’s begin. First in your functions.php, create a function:

function getAllRecordsFromRest(){
   //ADD ALL CODE IN HERE...
}

Inside your new function, is where we put the rest of our code.
Let’s figure out how many calls we need to make:

$totalItems = wp_count_posts()->publish;
$callsToMake = ceil($totalItems/100);

Let’s create an empty array and stuff it with the curl options:

$ch = array();
for($i=0;$i<$callsToMake;$i++){
    $page = $i + 1;
    $ch[$i] = curl_init(get_bloginfo('url').'/wp-json/wp/v2/posts?_embed&per_page=100&page='.$page);
    curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch[$i], CURLOPT_SSL_VERIFYPEER,  0);
}

As you can see above, we’re simply looping through the $callsToMake variable, to add a $page as part of our querystring. Similar to a paging system. As part of our loop, we are passing the options for curl through curl_setopt().

Now for the calls:

$mh = curl_multi_init();
        for($i=0;$i<$callsToMake;$i++){
            curl_multi_add_handle($mh, $ch[$i]);
        }
        $running = null;
        do {
            curl_multi_exec($mh, $running);
        } while ($running);
        for($i=0;$i<$callsToMake;$i++){
             if (!curl_errno($ch[$i])) {
                 $info = curl_getinfo($ch[$i]);
                 //error_log(print_r($info,true));
             }
            curl_multi_remove_handle($mh, $ch[$i]);
        }
        curl_multi_close($mh);
        $responses = array(); //array
        for($x=0;$x<$callsToMake;$x++){
            $responses[$x] = json_decode(curl_multi_getcontent($ch[$x]));
        }

First we create a handle $mh – this is what curl uses to execute and initialize the class. We loop through our $callstoMake once more to add the handles to curl_multi_add_handle().
We do a do while, and execute curl. Now if if there is an error in any of the calls, we will get them in curl_getinfo().

Finally, we get all of the contents of our calls through curl_multi_getcontent(), and we stuff it in our newly created $responses array.

Now our $responses will look something like:

$responses[$response[0], $response[1], $response[2]]

So we need to loop through them to create a big array of responses (in order).

So we do a final loop and do an array_push().

$final = array();
for($i=0;$i<count($responses);$i++){
    for($x=0;$x<count($responses[$i]);$x++){
       array_push($final,$responses[$i][$x] );
     }
}
return json_encode($final);

Then, we return our array as a JSON string by doing json_encode($final).

Now all we have to do is call our PHP function “getAllRecordsFromRest()” and we should have all of our posts.

2. Using JavaScript

We also have the option of using JavaScript for our multiple calls. Let’s start by getting the number of calls in our functions.php file:

function getNumCalls(){
    $totalItems = wp_count_posts('posts')->publish;
    $callsToMake = ceil($totalItems/100);
    return $callsToMake;
}

Now, in our header.php – let’s create a JavaScript variable that stores the number of calls to make – from the function that we created above:

<script>
    var numOfCalls = "<?php echo getNumCalls(); ?>";
</script>

Now, let’s create an array, that houses all of the responses, from our AJAX calls.

 var requests = Array();
 for(var i=0; i<numOfCalls; i++){
     var request = $.get('/wp-json/wp/v2/posts?_embed&per_page=100&page='+(i+1));
     requests.push(request);
 }

We then to a “.when()” to fire our calls. And doing a console.log() on the arguments, we see the responses. This technique is described in my previous post Working with Promise and Deferred.

$.when.apply($,requests).done(function(){
   console.log(arguments);
})

Using JS

Finally, let’s do a couple of for loops to merge the contents into one big array.

$.when.apply($,requests).done(function(){
    for(var i=0; i<arguments.length; i++){
       for(var x=0; x<arguments[i][0].length; x++){
            final.push(arguments[i][0][x]);
        }
    }
    console.log(final); //HERE ARE OUR RECORDS
}

And that should do it. Using either of these techniques should overcome the 100 record limitation of pulling records from WordPress REST api.
 

14 Comments

  1. I have a big problem with the last 3 steps because you are using jQuery and I only know vanilla js, can you make an updated in vanilla js? Thanks

    Reply
  2. thx for sharing this 🙂 works like a charm… but in my case i have change the “final” into “request” so it shoulb be “request.push(arguments[i][0][x]);” and i remove the first applied objects with “request.splice(0, numOfCalls);” and it works well. thx a lot Michael

    Reply
  3. thanks homie, this was exactly what i was looking for.
    i modified it for use on external sites, added an offset parameter and an error code check due to invalid page numbers.

    Reply
  4. Hi there. I *think* this is coming close to doing what I need. I’m using the PHP version. But I’m getting an error: “Warning: count(): Parameter must be an array or an object that implements Countable in {path} on line x”

    That line reads:
    for($x=0;$x<count($responses[$i]);$x++){

    Any ideas what could be going wrong here?

    Reply
      • Oh…hmmm…I hadn’t put that line above “$final = array()” thinking that it was just supposed to demonstrate what the result of $responses[$x] would be. When I put that line in, I get another error: “Parse error: syntax error, unexpected ‘,’, expecting ‘]'” Could it have to do with using a CPT in place of “post” for my curl_init url? I don’t think it does though, because I have also tried using just posts there instead, and I’m still getting the errors.

        Reply
          • It seems that that error for count() started in 7.2 – which I think is newer than this tutorial.
            Regardless, you can debug what you’re passing in count(). Try doing a var_dump() on the main array.

            Something like: var_dump($responses) – right before the for loops where the count() is. One of the subarrays are likely to be the culprit.

          • REST API is super new to me, so I apologize for all the questions. Your tutorial has come the closest to actually outputting anything, so that’s why I’m trying to dig into it to figure out what’s wrong. When I add the var_dump($responses), I get another error, from that same json URL: “array(1) { [0]=> NULL }
            Warning: count(): Parameter must be an array or an object that implements Countable in /path/”

  5. @laura – that’s it! the NULL is what’s causing the error. Try to figure out why the 1st array in $responses is NULL. Note that $responses is multi-dimensions, meaning its an array of arrays.

    Reply

Leave a Comment.