net-dl documentation

Download from a given URL, similar to wget or curl.

Whether the URL is downloaded as a file or it’s content is printed to stdout depends on the value of ‘Content-Type’ in the URL’s response headers.

CLI Usage

~$ # print content to stdout if text/html/json/xml
~$ net-dl 'https://httpbin.org/json'
{
   "slideshow": {
      "author": "Yours Truly",
      "date": "date of publication",
      "slides": [
         {
         "title": "Wake up to WonderWidgets!",
         "type": "all"
         },
         {
         "items": [
            "Why <em>WonderWidgets</em> are great",
            "Who <em>buys</em> WonderWidgets"
         ],
         "title": "Overview",
         "type": "all"
         }
      ],
      "title": "Sample Slide Show"
   }
}
~$ # save content to disk if file
~$ net-dl 'https://httpbin.org/image/svg'
[...................................                                     ]  50%
~$

Python module usage

>>> import net_dl
>>> url = 'https://httpbin.org/image/svg'
>>> dl = net_dl.Download(url)
>>> dl.get()
 [........................................................................] 100%
0
>>> import logging
>>> l = logging.get_logger()
>>> l.setLevel(logging.INFO)
>>> dl.get()
INFO:root:File already exists: /home/nate/g/net-dl/svg
0
>>>

See how to incorporate an external progress bar in the GitHub README.

The Download class

class net_dl.Download(url=None, destdir=getcwd(), destname=None, request_headers=None, chunk_size=None, progress_queue=None, remove_on_error=True, resume=None, timeout=config.HTTP_TIMEOUT, callback=None, callback_args=list(), callback_kwargs=dict())

The download task object.

It contains the source URL, the destination folder, and other needed attributes.

Variables:

• url – the source URL to download from

• destdir – the local destination folder

• resume – attempt to resume an incomplete download

get()

The typical way to start the download task.

Whether the URL is downloaded as a file or it’s content is printed to stdout depends on the value of ‘Content-Type’ in the URL’s response headers.

get_file(file_mode='wb')

Explicitly download the URL as a file.

This method is automatically used if Content-Type is “application-like”; progress bar is shown.

Variables:

file_mode – write mode for downloaded file. ‘ab’ is used to continue a partial download

get_text()

Explicitly download the URL’s text content.

This method is automatically used if Content-Type is “text-like”.