What Is cURL?

James Keenan

The automation and anonymity evangelist at Smartproxy. He believes in data freedom and everyone’s right to become a self-starter. James is here to share knowledge and help you succeed with residential proxies.

cURL is a command line tool that lets you transfer data from or to servers with URL syntax commands. Its name is the abbreviation of client URL. The cURL library lets you use over 20 protocols, including HTTP(S), FTP(S), POP3(S), and SMTP(S).

cURL supports proxies, FTP uploads, SSL connections, and even cookies. Since it is a command-line tool, it is syntax-dependent. That means your commands let you transfer data to or from servers without interaction.
what is curl
All new MacOS and Windows machines come with native cURL installed, so you do not need to put in any additional effort to start using cURL.

History of cURL

cURL was developed in 1997 by Daniel Stenberg as HttpGet. At first, it was a tool to get URLs, but soon the developers expanded its upload capabilities and added additional protocols.

Now, cURL is a client-side URL transfer library. On top of supporting all major protocols, cURL also works on most platforms, including Android, MS Windows, iOS, macOS, NetBSD or DOS. It supports over 40 programming languages and is compatible with IPv6.

Throughout the years cURL has become a global standard, which is installed in over 5 billion devices worldwide.

What can cURL do?

cURL can transfer data to and from servers through syntax commands. Since it doesn’t require any user interaction, cURL is often the first choice for automation.

Getting multiple URLs with cURL

cURL lets you get alphanumeric sequences by using brackets. Here’s an example:
http://www.website.example/photo[1-11].jpeg
http://www.website.example/photo[a-z].jpeg

{} brackets let you set multiple URLs, for example:
http://website.com/{home,about,contacts}

Pro tip: if you’re using cURL from a command line prompt, it’s best to put the whole URL into double quotes. Otherwise the shell might interfere with the syntax, especially if you use special symbols (e.g. ‘&’ or ‘?’) in the URL.

If you do not specify a protocol at the start of a syntax, cURL will default to using HTTP, but it will try other protocols based on the prefix.

What is cURL used for

cURL works with multiple programming languages, so developers use it to create various applications, but most people know cURL from calling APIs.

In our industry, cURL commands are great for checking whether a proxy connection is set up correctly.

How to start using cURL

On a Mac, launch Terminal and run a command ‘curl –man’ to pull up the “man pages”.

In Windows open PowerShell and run ‘curl –man’. This will pull up the “man pages”.

The usual cURL syntax looks like this:
cURL [option] [url]

For example, you can command cURL to show the output of a URL with:
cURL -v http://website.io

cURL can also use a proxy. For example:

cURL -x nl.smartproxy.com:10000 http://website.io

To learn more, check out the documentation in the cURL website. Also, check out our article about using cURL with a proxy.

cURL alternatives

If you find cURL to be too complex, you might want to try wget, which is a lot more simple and has less features.

If you are interested in accessing public data with a web scraper, visit our Scraping knowledge hub.