Eloquent get() vs chunk() vs lazy() benchmarks
[Updated August 2023]
As developers, our choices can greatly impact the performance and memory usage of our applications. Laravel offers us a variety of methods for data retrieval from databases, such as get(), chunk(), and lazy(). This article aims to explore these methods, highlighting their unique characteristics. We’ll compare them using some informal benchmarks to understand their implications on memory usage and speed. Note that these benchmarks are not scientific but should provide a useful comparative view.
Quick Summary
| Method | Description | Eager Load | Low Memory | Concurrent Safe |
|---|---|---|---|---|
get() |
Fetches all records at once, fastest method. | Yes | No | Yes |
chunk() |
Retrieves data in small “chunks,” conserving memory. | Yes | Yes | No |
cursor() |
Retrieves data one record at a time, saving memory. | No | Yes | Yes |
lazy() |
Fetches records in small segments using PHP generators, simplifying the syntax. | No | Yes | No |
Benchmarks
Please note that the benchmarks used in this article are not scientific and were conducted on a system running Laravel 8.74 and PHP 7.4.25 on macOS with Laravel Valet. All tests were conducted with Laravel 8.74 and PHP 7.4.25 on macOS with Laravel Valet.
10 000 records
| Method | Memory | Time |
|---|---|---|
get() |
26 MB | 261ms |
cursor() |
12 MB | 460ms |
lazy() |
9MB | 300ms |
chunk() by 500 |
6MB | 400ms |
100 000 records
| Method | Memory | Time |
|---|---|---|
get() |
277 MB | 2.72sec |
cursor() |
75 MB | 5.8sec |
lazy() |
9MB | 5.4sec |
chunk() by 500 |
6MB | 7.56sec |
Methods
get()
The get() method, employing the fetchAll() function, fetches all database records in a single operation. This method offers the highest speed among the alternatives; however, it comes at the cost of significant memory consumption. Thus, it’s best-suited for scenarios where you’re working with smaller datasets and where ample memory resources are available.
- Offers the fastest data retrieval
- Consumes a high amount of memory
chunk()
The chunk() method uses fetchAll() to retrieve data, similar to the get() method. However, instead of retrieving all records at once, it breaks the process down into smaller, manageable “chunks”. This approach dramatically reduces memory consumption, especially for larger datasets, by limiting the number of records retrieved at once based on a specified chunk size.
Despite being slower than get(), chunk() shines in scenarios where memory conservation is paramount. When performing operations that involve iterating and updating records simultaneously, consider using chunkById() to avoid potential issues, particularly when dealing with changes to primary or foreign keys.
- Best for memory conservation, especially with larger datasets
- Less speedy in comparison to
get() - Potential complications in concurrency. If you are updating the same records you are iterating over (within the chunk operation), you may run into issues.
cursor()
The cursor() method leverages PHP´s fetch() function to retrieve records from the database buffer one at a time. As a result, it uses less memory than get(), making it more efficient for handling larger datasets. However, the trade-off is speed; it’s slower due to its one-by-one record iteration. An important advantage of cursor() is its consistency in processing datasets that might be subject to change during the processing period.
- Effective memory usage - ideal for large datasets
- Consistently processes datasets even when data changes during the operation
- Slower than
get() - Unable to handle eager loading of relationships
- There’s a potential for memory exhaustion depending on the buffer size, particularly with extremely large collections
lazy()
Introduced in Laravel 8, the lazy() method is a more syntax-friendly version of chunk(). Like chunk(), lazy() retrieves records in small segments. However, instead of requiring a callback, lazy() employs PHP generators. This results in a simplified syntax, as it returns a LazyCollection for your convenience.
The lazy() method strikes a balance between efficient memory use and readability. While handling larger datasets and performing operations that involve iterating and updating records simultaneously, you can use lazyById() to maintain data consistency, especially when updating primary or foreign keys.
- Efficient memory usage similar to
chunk() - Provides a cleaner syntax due to PHP generators
- Slower than
get() - Unable to handle eager loading of relationships
- Possible concurrency Issues: If the underlying data changes during the iteration process, there might be inconsistencies in the results.
- Suited for operations that require simultaneous iteration and updates on records, using
lazyById()
Conclusion
As we’ve explored, Laravel offers a variety of methods for data retrieval, each with its own unique strengths and weaknesses. The get() and chunk() methods, perhaps the most commonly used, offer distinct benefits. get(), while fast and capable of eager loading, consumes substantial memory, which may limit its utility with large datasets. On the other hand, chunk() method allows for memory efficiency with larger datasets and supports eager loading, but isn’t safe for operations where data changes concurrently.
Less commonly used but still powerful, the cursor() method stands out for its memory efficiency and concurrent safety, despite being slower and lacking support for eager loading. Finally, lazy(), the newcomer to the Laravel ecosystem, provides a balance of chunk()’s memory efficiency and simpler syntax but, like cursor(), does not support eager loading and isn’t safe with concurrently changing data.
Ultimately, the choice of method should be guided by your application’s specific needs:
- Size of your data
- Available system memory
- Requirement for eager loading
- Whether the data is likely to change during the operation.
While get() and chunk() are popular for a reason, every method has its place, and the optimal choice always depends on the context. Equipped with an understanding of each method’s trade-offs, you’re now prepared to make an informed decision that best optimizes your application’s performance and memory usage.