CURL

`1. Introduction to cURL`

visit the official cURL documentation at https://curl.se/docs/` or use man curl in your terminal`

cURL (Client URL) is a command-line tool and library for transferring data with URLs. It supports numerous protocols including HTTP, HTTPS, FTP, FTPS, SFTP, and many more. It’s one of the most powerful tools for testing APIs, downloading files, and automating web requests.

Why Use cURL?

Testing APIs without writing code
Automating downloads and uploads
Debugging web services
Scripting repetitive tasks
Cross-platform compatibility
Lightweight and fast

`2. Installation`

Check if cURL is Already Installed

curl --version

`3. Basic Syntax and Usage`

Basic Syntax

curl [options] [URL]

Simple GET Request

curl https://api.example.com/users

Common Options

Option	Description
`-X, --request`	Specify HTTP method
`-H, --header`	Add custom header
`-d, --data`	Send data in request body
`-o, --output`	Write output to file
`-O`	Save with remote filename
`-i, --include`	Include response headers
`-I, --head`	Show headers only
`-v, --verbose`	Verbose output
`-s, --silent`	Silent mode
`-L, --location`	Follow redirects
`-u, --user`	Server authentication

`4. HTTP Methods`

GET Request (Default)

curl https://api.example.com/users
# or explicitly
curl -X GET https://api.example.com/users

POST Request

curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d '{"name":"John Doe","email":"john@example.com"}'

PUT Request

curl -X PUT https://api.example.com/users/123 \
  -H "Content-Type: application/json" \
  -d '{"name":"Jane Doe"}'

PATCH Request

curl -X PATCH https://api.example.com/users/123 \
  -H "Content-Type: application/json" \
  -d '{"email":"newemail@example.com"}'

DELETE Request

curl -X DELETE https://api.example.com/users/123

HEAD Request (Headers Only)

curl -I https://api.example.com/users

OPTIONS Request

curl -X OPTIONS https://api.example.com/users -i

`5. Headers and Authentication`

Adding Custom Headers

curl -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  https://api.example.com/data

Multiple Headers

curl -H "User-Agent: MyApp/1.0" \
  -H "Accept: application/json" \
  -H "X-Custom-Header: value" \
  https://api.example.com/data

Basic Authentication

curl -u username:password https://api.example.com/protected

Bearer Token Authentication

curl -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  https://api.example.com/protected

API Key Authentication

curl -H "X-API-Key: YOUR_API_KEY" \
  https://api.example.com/data

Digest Authentication

curl --digest -u username:password https://api.example.com/protected

`6. Request Data and Body`

Sending JSON Data

curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d '{"name":"John","age":30}'

Sending JSON from File

curl -X POST https://api.example.com/users \
  -H "Content-Type: application/json" \
  -d @data.json

Sending Form Data

curl -X POST https://api.example.com/form \
  -d "name=John" \
  -d "email=john@example.com"

Sending URL-Encoded Data

curl -X POST https://api.example.com/form \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "name=John&email=john@example.com"

Uploading Files

curl -X POST https://api.example.com/upload \
  -F "file=@/path/to/file.pdf" \
  -F "description=Important document"

Multiple File Upload

curl -X POST https://api.example.com/upload \
  -F "file1=@image1.jpg" \
  -F "file2=@image2.jpg"

Query Parameters

curl "https://api.example.com/search?q=curl&limit=10&page=1"

`7. Response Handling`

Save Response to File

curl https://api.example.com/data -o response.json

Save with Remote Filename

curl -O https://example.com/file.pdf

Include Response Headers

curl -i https://api.example.com/users

Show Only Headers

curl -I https://api.example.com/users

Show HTTP Status Code

curl -o /dev/null -s -w "%{http_code}\n" https://api.example.com/users

Pretty Print JSON (with jq)

curl https://api.example.com/users | jq '.'

Extract Specific Fields (with jq)

curl https://api.example.com/users | jq '.data[].name'

Write Headers to Separate File

curl -D headers.txt https://api.example.com/data -o response.json

`8. Advanced Features`

Follow Redirects

curl -L https://short.url/redirect

Limit Redirect Follows

curl -L --max-redirs 5 https://example.com

Set Timeout

curl --connect-timeout 10 --max-time 30 https://api.example.com/data

Retry on Failure

curl --retry 3 --retry-delay 5 https://api.example.com/data

Use Proxy

curl -x http://proxy.example.com:8080 https://api.example.com/data

Proxy with Authentication

curl -x http://proxy.example.com:8080 \
  -U proxyuser:proxypass \
  https://api.example.com/data

Ignore SSL Certificate Errors (Not Recommended for Production)

curl -k https://self-signed.example.com

Specify SSL/TLS Version

curl --tlsv1.2 https://api.example.com/data

Custom DNS Resolution

curl --resolve example.com:443:192.168.1.1 https://example.com

Rate Limiting

curl --limit-rate 100K https://example.com/largefile.zip

# Save cookies
curl -c cookies.txt https://example.com/login
 
# Send cookies
curl -b cookies.txt https://example.com/dashboard
 
# Both save and send
curl -b cookies.txt -c cookies.txt https://example.com/page

Session Management

# Login and save session
curl -X POST https://api.example.com/login \
  -d '{"username":"user","password":"pass"}' \
  -c session.txt
 
# Use session for subsequent requests
curl -b session.txt https://api.example.com/protected

Range Requests (Partial Download)

curl -r 0-999 https://example.com/largefile.zip

Resume Download

curl -C - -O https://example.com/largefile.zip

Multiple URLs

curl https://example.com/file1.txt https://example.com/file2.txt

URL Globbing

curl https://example.com/file[1-10].txt
curl https://example.com/{file1,file2,file3}.txt

`9. Debugging and Troubleshooting`

Verbose Output

curl -v https://api.example.com/data

Trace ASCII

curl --trace-ascii trace.txt https://api.example.com/data

Trace Binary

curl --trace trace.bin https://api.example.com/data

Show Request and Response Timing

curl -w "\nTime Total: %{time_total}s\nTime Connect: %{time_connect}s\n" \
  https://api.example.com/data

Detailed Timing Information

curl -w "@curl-format.txt" -o /dev/null -s https://api.example.com/data

Create curl-format.txt:

    time_namelookup:  %{time_namelookup}s\n
       time_connect:  %{time_connect}s\n
    time_appconnect:  %{time_appconnect}s\n
   time_pretransfer:  %{time_pretransfer}s\n
      time_redirect:  %{time_redirect}s\n
 time_starttransfer:  %{time_starttransfer}s\n
                    ----------\n
         time_total:  %{time_total}s\n

Show Only Errors

curl -s -S https://api.example.com/data

Test Connection Only

curl --head --silent --connect-timeout 5 https://example.com > /dev/null && echo "Success" || echo "Failed"

`10. Practical Examples`

Testing REST API

# Create a resource
curl -X POST https://jsonplaceholder.typicode.com/posts \
  -H "Content-Type: application/json" \
  -d '{"title":"Test","body":"Content","userId":1}'
 
# Get all resources
curl https://jsonplaceholder.typicode.com/posts
 
# Get specific resource
curl https://jsonplaceholder.typicode.com/posts/1
 
# Update resource
curl -X PUT https://jsonplaceholder.typicode.com/posts/1 \
  -H "Content-Type: application/json" \
  -d '{"title":"Updated","body":"New content","userId":1}'
 
# Delete resource
curl -X DELETE https://jsonplaceholder.typicode.com/posts/1

Downloading Files

# Download single file
curl -O https://example.com/file.pdf
 
# Download with custom name
curl https://example.com/file.pdf -o myfile.pdf
 
# Download multiple files
curl -O https://example.com/file1.pdf -O https://example.com/file2.pdf
 
# Resume interrupted download
curl -C - -O https://example.com/largefile.zip

Web Scraping

# Get page content
curl https://example.com
 
# Get specific element (with grep)
curl -s https://example.com | grep -o '<title>.*</title>'
 
# Save HTML page
curl https://example.com -o page.html

Testing GraphQL API

curl -X POST https://api.example.com/graphql \
  -H "Content-Type: application/json" \
  -d '{"query":"{ users { id name email } }"}'

Checking Website Status

# Simple health check
curl -Is https://example.com | head -1
 
# Detailed status check
curl -o /dev/null -s -w "HTTP Status: %{http_code}\nTotal Time: %{time_total}s\n" https://example.com

Sending Webhooks

curl -X POST https://hooks.slack.com/services/YOUR/WEBHOOK/URL \
  -H "Content-Type: application/json" \
  -d '{"text":"Hello from cURL!"}'

Testing POST with Form Data

curl -X POST https://httpbin.org/post \
  -F "name=John Doe" \
  -F "email=john@example.com" \
  -F "file=@document.pdf"

OAuth Token Request

curl -X POST https://oauth.example.com/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id=YOUR_CLIENT_ID" \
  -d "client_secret=YOUR_CLIENT_SECRET"

`11. Best Practices`

Security Best Practices

Never include sensitive data in URLs (use POST body instead)

# Bad
curl "https://api.example.com/login?password=secret123"
 
# Good
curl -X POST https://api.example.com/login \
  -d '{"password":"secret123"}'

Use environment variables for secrets

export API_TOKEN="your_secret_token"
curl -H "Authorization: Bearer $API_TOKEN" https://api.example.com/data

Validate SSL certificates (avoid -k in production)

# Production
curl https://api.example.com/data
 
# Development only
curl -k https://dev.example.com/data

Performance Best Practices

Use compression

curl --compressed https://api.example.com/data

Reuse connections

curl --keepalive-time 60 https://api.example.com/data

Set appropriate timeouts

curl --connect-timeout 5 --max-time 30 https://api.example.com/data

Scripting Best Practices

Check HTTP status codes

response=$(curl -s -w "\n%{http_code}" https://api.example.com/data)
http_code=$(echo "$response" | tail -n1)
body=$(echo "$response" | sed '$d')
 
if [ "$http_code" -eq 200 ]; then
  echo "Success: $body"
else
  echo "Error: HTTP $http_code"
fi

Handle errors gracefully

curl -f -s https://api.example.com/data || echo "Request failed"

Use configuration files

# .curlrc file
user-agent = "MyApp/1.0"
connect-timeout = 10
max-time = 30
 
# Use it
curl -K .curlrc https://api.example.com/data

Debugging Best Practices

Use verbose mode during development

curl -v https://api.example.com/data

Save requests for reproduction

curl https://api.example.com/data --trace-ascii debug.txt

Test incrementally

# First, test connection
curl -I https://api.example.com
 
# Then test auth
curl -H "Authorization: Bearer TOKEN" https://api.example.com/data
 
# Finally, test full request
curl -X POST -H "Authorization: Bearer TOKEN" -d '...' https://api.example.com/data

`Common Use Cases Cheat Sheet`

# Simple GET
curl https://api.example.com/users
 
# POST JSON
curl -X POST https://api.example.com/users -H "Content-Type: application/json" -d '{"name":"John"}'
 
# With auth
curl -H "Authorization: Bearer TOKEN" https://api.example.com/data
 
# Download file
curl -O https://example.com/file.pdf
 
# Follow redirects
curl -L https://short.url
 
# Save response
curl https://api.example.com/data -o response.json
 
# Upload file
curl -F "file=@document.pdf" https://api.example.com/upload
 
# Verbose output
curl -v https://api.example.com/data
 
# Show only status
curl -o /dev/null -s -w "%{http_code}" https://api.example.com
 
# With timeout
curl --connect-timeout 10 --max-time 30 https://api.example.com/data

Vault 📚

Explorer

1. Introduction to cURL