Performance
The performance of the plugin is highly dependent on various external factors:
Latency and bandwidth to AWS
Network infrastructure
FD Host hardware
FD Load
Ratio number of elements/size
And many more.
In short, it is not possible to establish an exact reference for the duration required to complete a backup.
Typically, backups that do not use the export and download functions will proceed as quickly as AWS permits However, when using the export and download functions, the aforementioned factors become increasingly pertinent, as certain files will be created in S3 that must subsequently be downloaded.
As a guideline concerning the number of records per table:
Numerous small tables to protect: More files (one file per table) per time period, but smaller speed (MB/s).
Large tables to protect: Fewer files (one file per table) per time period, but higher speed (MB/s).
It is also important to note that the snapshot cleanup process requires some time to finalize, in addition to the export operation itself.
It is recommended to benchmark your own environment based on your specific requirements and needs.
There are various strategies available to use this plugin, so it is recommended to evaluate which option best meets your needs prior to deploying the jobs across your entire environment, ensuring optimal results:
You can have a job per instance or cluster (recommended)
You can have multiple instances per job or even all your instances or clusters in the same job (not recommended)
You can split your workload through a schedule, or try to run all your jobs together
You can specifically select the instances you want to backup or backup them all
You can specifically select the clusters you want to backup or backup them all
You can specifically select the tables you want to export and backup or process them all
You can run your File Daemon on premise or in the cloud
You can use default internet connection to AWS or use a dedicated AWS connection (https://aws.amazon.com/directconnect/)
You can run your Storage Daemon and File Daemon in the same host, so you skip one network hop in the process (recommended).
Go back to: Best Practices.