Yes, indeed. The list operation is expensive. The S3 spec says that the list output needs to be sorted.
1. All filenames are read.
2. All filenames are sorted.
3. Pagination applied.
It doesn't scale obviously, but works ok-ish for a smaller data set. It is difficult to do this efficiently without introducing complexity. My applications don't use listing, so I prioritised simplicity over performance for the list operation.
espenb|16 days ago
1. All filenames are read. 2. All filenames are sorted. 3. Pagination applied.
It doesn't scale obviously, but works ok-ish for a smaller data set. It is difficult to do this efficiently without introducing complexity. My applications don't use listing, so I prioritised simplicity over performance for the list operation.
luke5441|16 days ago
Listing was IMO a problem with minio as well, but maybe it is not that important because it seems to have succeeded anyway.