The AWS CLI makes working with files in S3 very easy. However, the file globbing available on most Unix/Linux systems is not quite as easy to use with the AWS CLI. S3 doesn’t have folders, but it does use the concept of folders by using the “/” character in S3 object keys as a folder delimiter.

To copy all objects in an S3 bucket to your local machine simply use the aws s3 cp command with the --recursive option.

For example aws s3 cp s3://big-datums-tmp/ ./ --recursive will copy all files from the “big-datums-tmp” bucket to the current working directory on your local machine. If there are folders represented in the object keys (keys containing “/” characters), they will be downloaded as separate directories in the target location.

The command aws s3 cp s3://big-datums-tmp/myFolder/ ./ --recursive is almost the same as the one above, but this command will only copy files from “myFolder” folder (objects with keys starting with “myFolder/”).

For using wildcards and patterns to copy only certain files, refer to Using Wildcards with AWS CLI on how to correctly use the --include and --exclude options.