| Age | Commit message (Collapse) | Author |
|---|
|
Previously, every file downloaded in a batch would create a new Site
object. Now, the Site object created by the first file will be reused
by subsequent files if it matches the file's requested host, which will
significantly speed up the download process, assuming all files are from
the same site. This is a quick and dirty fix which could be improved to
better handle situations where there are a mix of files from different
sites.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Defer string formatting until needed by removing f-strings in log
messages.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
When the input is an invalid URL, the error message will report the
whole input string instead of a URL fragment.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Also, use a LoggerAdapter to add contextual info (such as filenames)
to log messages when downloading, especially useful with threaded
batch processing.
|
|
|
|
|
|
An invalid line in the batch file would cause the last valid file to
be downloaded twice.
|
|
|
|
|
