Discovering PHP's yield Keyword After 10 Years

11th March 2025

I started learning PHP in 2014 and working as a full-time web developer not long after, so at this point I've been doing it for a little over a decade. It wasn't until today I learned about yield.

Today I needed to write a command to iterate over files stored in an S3 bucket and process each of them. We can list the contents of the bucket using the listObjectsV2 method that calls the S3 API;

1AwsFacade::createClient('s3', $config)
2    ->listObjectsV2([
3        'Bucket' => 'top-secret-bucket'
4    ]);

This API call will return up to a thousand results, but given that we're scanning a bucket with hundreds of thousands of files, we're going to need to make this call hundreds of times. When paginating over database records this can be done easily using LIMIT and OFFSET, but what about when making HTTP calls like the one to S3?

The S3 API allows you to fetch subsequent pages of results by passing in a ContinuationToken, which is given back with each set of results. This means we can do something like the following;

 1public function handle(): void
 2{
 3    $s3Client = AwsFacade::createClient('s3', $config);
 4    $continuationToken = null;
 5 
 6    do {
 7        $response = $s3Client->listObjectsV2([
 8            'Bucket' => 'top-secret-bucket',
 9            'ContinuationToken' => $continuationToken,
10        ]);
11 
12        foreach ($response->get('Contents') as $result) {
13            $this->processResult($result);
14        }
15 
16        $continuationToken = $response->get('NextContinuationToken');
17        $isTruncated = $response->get('IsTruncated');
18    } while ($isTruncated);
19}

IsTruncated indicates that there are more results to come and NextContinuationToken gives us a pointer we can use to paginate over all the files in the bucket.

However; fetching the list of files from S3 is only part of what I need to accomplish, so I don't really want all the pagination functionality cluttering the main method of my command. What I would prefer is something like;

1public function handle(): void
2{
3    foreach ($this->listS3Files() as $s3File) {
4        $this->processFile($s3File);
5    }
6}

One way of doing this would be to move the do-while loop to a separate method and build an array of results that gets passed back, but when we're working with so many records that becomes impractical. We need some way of extracting our HTTP call to a separate method, while easily paginating the results so we can make as many calls as necessary. Enter yield.

 1public function handle(): void
 2{
 3    $this->s3Client = AwsFacade::createClient('s3', $config);
 4 
 5    foreach ($this->listS3Files() as $s3File) {
 6        $this->processFile($s3File);
 7    }
 8}
 9 
10// The return type of the method is now iterable 
11private function listS3Files(): iterable
12{
13    $continuationToken = null;
14 
15    do {
16        $response = $this->s3Client->listObjectsV2([
17            'Bucket' => 'top-secret-bucket',
18            'ContinuationToken' => $continuationToken,
19        ]);
20 
21        foreach ($response->get('Contents') as $result) {
22            // Using yield rather than return allows us to paginate over results 
23            yield $result;
24        }
25 
26        $continuationToken = $response->get('NextContinuationToken');
27        $isTruncated = $response->get('IsTruncated');
28    } while ($isTruncated);
29}

Replacing return with yield changes this method into what's called a Generator. When a method returns a result, everything else within the scope of the method is discarded. If we yield the result instead, the state of the generator is saved, $continuationToken is preserved and it picks up where it left off with each call, lazily paginating over all the results. This means we can use the method in our foreach loop and iterate over each file in the S3 bucket, processing each as necessary, while making additional HTTP calls to fetch more results whenever we need them!

It surprises me that in all these years I've never needed to accomplish anything like this before, but now I have an extra tool on my toolkit for whenever the need arises again! You can learn more about the yield keyword on the PHP documentation page about Generators.