Back to blog

How to Use cURL in JavaScript: Fetch, Axios, and Best Practices

Share article:

Running cURL in JavaScript means translating a terminal cURL command into a JavaScript or Node.js HTTP request using native APIs or external libraries. If you need to migrate a working terminal cURL command into your codebase, you have options. You can execute the system cURL binary directly using child_process or node-libcurl, or rewrite it natively using Fetch or Axios. This guide provides a flag-by-flag cURL-to-JS translation guide and a decision framework to help you choose the best approach for your stack.

A graphic illustrating the concept of converting terminal cURL commands into JavaScript and Node.js code snippets.

TL;DR

  • Fetch is the built-in JavaScript equivalent of cURL and works in both browsers and Node.js 18+ with no dependencies
  • Axios is the recommended default for Node.js projects thanks to automatic JSON parsing, better error handling, and built-in timeouts
  • Running cURL via child_process works, but it's best kept for quick scripts or when you already have a working command
  • node-libcurl gives you full libcurl power, but it's a niche choice for advanced networking needs
  • Translating cURL to JavaScript is a flag-by-flag mapping, not a one-to-one copy-and-paste
  • Proxies are essential for scraping at scale, and neither Fetch nor Axios supports them natively without extra setup
  • If you're dealing with CAPTCHAs or heavy bot protection, switching to a scraping API is more practical than tweaking HTTP clients

What "cURL in JavaScript" actually means

Before writing any code, it helps to understand the split. cURL is a command-line tool. It runs on your system, not in a browser. You can't call it from browser-side JavaScript – browsers simply don't allow your code to run system programs.

The answer depends on where your JavaScript runs.

In the browser, you use the Fetch API. It can do the same things cURL does – send GET and POST requests, set custom headers, and include a request body. The one catch is CORS. If you're making a request to a different domain, the server on the other end has to allow it explicitly. A cURL command might fail in the browser, not because your code is wrong, but because the server never said, "yes, other websites can call me." There's no client-side workaround for this.

In Node.js, you have two options. You can run the actual cURL program from your script using the built-in child_process module – basically telling Node.js "run this terminal command for me and give me the result." Or you can skip cURL entirely and use a JavaScript HTTP library (Fetch, Axios, or node-libcurl) that makes the same kinds of requests without needing cURL installed at all.

The practical takeaway: for almost every JavaScript project, using a built-in HTTP library is the better choice. It works everywhere your code runs, it's easier to test, and it doesn't break when someone deploys to a server that doesn't have cURL installed. Shelling out to cURL has its place, but it's the exception.

If you're coming from a cURL-heavy workflow, it's worth brushing up on how cURL GET requests work and how cURL handles proxies before diving into the JavaScript equivalents below.

Running system cURL from Node.js with child_process

The most literal way to use cURL in JavaScript is to tell Node.js to run a cURL command in the terminal and return the output. This can be useful when migrating a shell script to Node.js, automating a quick task, or reusing an existing cURL command that isn't worth translating to Fetch or Axios.

Basic GET request with exec()

The child_process module is built into Node.js, so no installation is required. ts exec() function runs a shell command and returns the output through a callback.

const { exec } = require('child_process');
exec('curl -s https://jsonplaceholder.typicode.com/posts/1', (error, stdout, stderr) => {
  if (error) {
    console.error('Command failed:', error.message);
    return;
  }
  if (stderr) {
    console.error('cURL error output:', stderr);
    return;
  }
  const data = JSON.parse(stdout);
  console.log(data.title);
});

The -s flag tells cURL to run silently, preventing progress information from being mixed into the response. exec() launches the command in a child shell process and gives you stdout, stderr, and any execution error. Since the response is returned as a string, you need to parse JSON yourself.

For simple scripts that don't need asynchronous execution, you can also use execSync(). It returns the command output directly but blocks the Node.js process until cURL finishes, making it a poor fit for servers and other concurrent applications.

Safer execution with spawnSync()

As cURL commands gain more headers, options, or dynamic values, building them as a single shell string becomes harder to manage and can introduce shell-injection risks. spawnSync() lets you provide each argument separately instead.

const { spawnSync } = require('child_process');
const result = spawnSync('curl', [
  '-s',
  '-X', 'GET',
  '-H', 'Accept: application/json',
  '-H', 'User-Agent: my-node-script/1.0',
  'https://jsonplaceholder.typicode.com/posts?_limit=3'
]);
if (result.error) {
  console.error('Failed to run cURL:', result.error.message);
} else if (result.status !== 0) {
  console.error('cURL exited with code:', result.status);
  console.error(result.stderr.toString());
} else {
  const posts = JSON.parse(result.stdout.toString());
  posts.forEach(post => console.log(`- ${post.title}`));
}

Because each argument is passed separately rather than interpreted as part of a shell command, spawnSync() is safer when working with dynamic input. The asynchronous spawn() function follows the same argument-based approach without blocking the Node.js process.

When this approach falls apart

Running cURL through child_process works, but it comes with several drawbacks:

  • cURL must be installed. Minimal Docker images, serverless environments, and CI systems may not include the binary.
  • Responses are raw output. You need to parse response bodies and handle unexpected formats yourself.
  • Testing is harder. Mocking child processes is generally less convenient than mocking an HTTP client.
  • HTTP error handling is clunky. Status codes aren't exposed automatically, so you need additional cURL options and parsing.

For quick scripts or migrations, calling cURL from Node.js can be practical. For most application code, native Fetch or libraries such as Axios provide cleaner request handling without relying on an external cURL process.

Skip the boilerplate

Decodo's Web Scraping API handles proxies, CAPTCHAs, and anti-bot detection so your code stays short and your requests actually land.

Using node-libcurl for direct libcurl bindings

If child_process runs cURL from JavaScript, node-libcurl puts cURL's underlying engine inside JavaScript. It's a native addon that wraps libcurl and exposes its features directly to Node.js without running shell commands.

This is a specialist tool. For most projects, Fetch or Axios is simpler. node-libcurl becomes useful when you need low-level TLS configuration, custom cipher suites, CURLOPT_* options, or multi-handle concurrency for high-throughput scraping.

Installation

Install node-libcurl with:

npm install node-libcurl

Because node-libcurl is a native addon, installation may require node-gyp and a C++ build toolchain. This can require additional setup in environments such as CI pipelines or Docker containers.

GET request with the curly interface

The simplest node-libcurl API is curly, a convenience wrapper that supports async/await and feels similar to a modern JavaScript HTTP client.

const { curly } = require('node-libcurl');
async function getPost() {
  try {
    const { statusCode, data } = await curly.get(
      'https://jsonplaceholder.typicode.com/posts/1'
    );
    console.log('Status:', statusCode);
    console.log('Title:', data.title);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
getPost();

curly automatically parses JSON responses, so data is already an object, while statusCode contains the HTTP status code.

Using the Curl class for fine-grained control

The lower-level Curl class is where node-libcurl becomes particularly useful. It gives you direct access to libcurl options for requests that need more control than Fetch or Axios typically provide.

const { Curl, CurlCode } = require('node-libcurl');
const curl = new Curl();
curl.setOpt('URL', 'https://jsonplaceholder.typicode.com/posts/1');
curl.setOpt('FOLLOWLOCATION', true);
curl.setOpt('TIMEOUT', 10);
curl.setOpt('HTTPHEADER', [
  'Accept: application/json',
  'User-Agent: my-node-script/1.0',
]);
curl.on('end', (statusCode, body, headers) => {
  console.log('Status:', statusCode);
  console.log('Title:', JSON.parse(body).title);
  curl.close();
});
curl.on('error', (error, errorCode) => {
  console.error('Request failed:', error.message);
  console.error('cURL error code:', errorCode);
  curl.close();
});
curl.perform();

Options set through curl.setOpt() map closely to libcurl's CURLOPT_* functionality. This gives you access to features such as custom DNS resolution, specific TLS versions, proxy tunneling, and detailed connection settings.

If you don't need that level of control, Fetch or Axios will usually be easier to install, read, and maintain.

Fetch API: The native JavaScript HTTP client

If you're starting a new project and wondering which HTTP client to use, start here. Fetch is built into every modern browser and comes with Node.js 18+ as a global – no packages, no imports, no npm install. It's just there.

For most developers looking for a cURL equivalent in JavaScript, Fetch is the answer. It won't do everything cURL can, but it covers the vast majority of use cases with zero dependencies.

Basic GET request

async function getPost() {
  try {
    const response = await fetch('https://jsonplaceholder.typicode.com/posts/1');
    const data = await response.json();
    console.log(data.title);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
getPost();

Two lines to make a request and parse the response. That's it. No modules to require, no clients to instantiate. If you're on Node.js 16 or below, you'll need the node-fetch package to get the same API, but from Node.js 18 onward, fetch() is global just like it is in the browser.

POST request with a JSON body

Sending data works the same way a cURL POST does. You specify the method, set the content type, and pass the body. The syntax is just more verbose than a one-liner in the terminal.

async function createPost() {
  try {
    const response = await fetch('https://jsonplaceholder.typicode.com/posts', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        title: 'Test post',
        body: 'Sent via Fetch',
        userId: 1,
      }),
    });
    const data = await response.json();
    console.log('Created post ID:', data.id);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
createPost();

One thing that trips people up: the body must be a string. You can't pass a plain JavaScript object, and you need JSON.stringify() every time. Axios handles this automatically, which is one reason people reach for it instead.

Custom headers

The headers option takes a plain object. Each key-value pair is equivalent to an -H flag in cURL.

const response = await fetch('https://api.github.com/user', {
  headers: {
    'Authorization': 'Bearer ghp_your_token_here',
    'Accept': 'application/vnd.github.v3+json',
    'User-Agent': 'my-node-script/1.0',
  },
});

You can also use the headers constructor if you need to build headers programmatically, but for most cases, the plain object works fine.

Handling errors

Here's the thing about Fetch that catches almost everyone the first time: it doesn't throw on HTTP errors. A 404, a 500, a 403 – Fetch considers all of these successful responses because the server did respond. It only throws on actual network failures, like the server being unreachable.

async function getPost() {
  try {
    const response = await fetch('https://jsonplaceholder.typicode.com/posts/9999');
    if (!response.ok) {
      console.error(`Server returned ${response.status}: ${response.statusText}`);
      return;
    }
    const data = await response.json();
    console.log(data.title);
  } catch (error) {
    // This only fires on network errors, not HTTP errors
    console.error('Network error:', error.message);
  }
}
getPost();

Always check the response.ok before parsing the body. If you skip this, you'll eventually try to JSON.parse() an HTML error page and spend 20 minutes wondering why your data is undefined.

Timeouts

cURL has --max-time. Axios has a timeout option. Fetch has... nothing. By default, a Fetch request will hang indefinitely if the server never responds. You need to wire up an AbortController yourself.

async function fetchWithTimeout(url, timeoutMs = 5000) {
  const controller = new AbortController();
  const timeoutId = setTimeout(() => controller.abort(), timeoutMs);
  try {
    const response = await fetch(url, { signal: controller.signal });
    clearTimeout(timeoutId);
    if (!response.ok) {
      throw new Error(`HTTP ${response.status}`);
    }
    return await response.json();
  } catch (error) {
    clearTimeout(timeoutId);
    if (error.name === 'AbortError') {
      throw new Error(`Request timed out after ${timeoutMs}ms`);
    }
    throw error;
  }
}

It's not a lot of code, but it's code you have to write every time, or abstract into a helper. This is one of the biggest practical arguments for Axios over raw Fetch in production scraping work.

CORS in the browser

As mentioned earlier, browser-side fetch() requests are subject to CORS restrictions. If CORS prevents a direct request, you’ll need to make it server-side instead, for example through a Node.js backend or serverless function.

For web scraping with JavaScript, this generally isn’t an issue when the scraping logic runs server-side in Node.js.

Axios: the developer-friendly HTTP library

Security note: On March 31, 2026, compromised npm credentials were used to publish malicious Axios versions 1.14.1 and 0.30.4. If you installed either version, revert to 1.14.0 or 0.30.3 and rotate any credentials that may have been exposed.

With that out of the way, Axios is still one of the most popular HTTP libraries in the JavaScript ecosystem for good reason. It works in both Node.js and the browser, parses JSON automatically, has built-in timeout support, and gives you cleaner error handling than raw Fetch. For Node.js projects that need more ergonomics than fetch() offers, it's been the go-to choice for years.

Installation

npm install axios@1.14.0

We're pinning to 1.14.0 here deliberately.

Basic GET request

const axios = require('axios');
async function getPost() {
  try {
    const response = await axios.get(
      'https://jsonplaceholder.typicode.com/posts/1'
    );
    console.log(response.data.title);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
getPost();

Notice what's missing compared to Fetch: no .json() call. Axios detects the Content-Type header and parses the response body automatically. response.data is already a JavaScript object. It's a small thing, but it adds up over hundreds of requests.

POST request

Sending a POST request is similarly streamlined. Pass a JavaScript object directly, and Axios serializes it to JSON and sets the Content-Type header for you.

const axios = require('axios');
async function createPost() {
  try {
    const response = await axios.post(
      'https://jsonplaceholder.typicode.com/posts',
      {
        title: 'Test post',
        body: 'Sent via Axios',
        userId: 1,
      }
    );
    console.log('Created post ID:', response.data.id);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
createPost();

No JSON.stringify(), no manual Content-Type header. Compare this to the Fetch version of the same request, and you'll see why people reach for Axios.

Custom headers

Setting headers works like cURL's -H flag – pass them as an object in the config parameter.

const axios = require('axios');
async function getGitHubUser() {
  try {
    const response = await axios.get('https://api.github.com/user', {
      headers: {
        'Authorization': 'Bearer ghp_your_token_here',
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'my-node-script/1.0',
      },
    });
    console.log(response.data.login);
  } catch (error) {
    console.error('Request failed:', error.message);
  }
}
getGitHubUser();

Timeout

One of Axios's biggest practical advantages over Fetch is the timeout:

const response = await axios.get('https://slow-api.example.com/data', {
  timeout: 5000, // 5 seconds
});

If the server doesn't respond within the limit, Axios throws an error with code: 'ECONNABORTED'.

Error handling

Unlike Fetch, Axios actually throws errors on 4xx and 5xx responses. This means your catch block handles both network failures and HTTP errors, which is usually what you want.

const axios = require('axios');
async function getPost() {
  try {
    const response = await axios.get(
      'https://jsonplaceholder.typicode.com/posts/9999'
    );
    console.log(response.data);
  } catch (error) {
    if (error.response) {
      // Server responded with a non-2xx status
      console.error('Status:', error.response.status);
      console.error('Body:', error.response.data);
    } else if (error.request) {
      // Request was sent, but no response received
      console.error('No response from server');
    } else {
      // Something went wrong setting up the request
      console.error('Setup error:', error.message);
    }
  }
}
getPost();

The three-tier structure (error.responseerror.requesterror.message) covers every failure mode. You'll know exactly where things went wrong without parsing status codes out of a raw string.

Interceptors

This is where Axios pulls ahead for any project with more than a handful of requests. Interceptors let you run logic on every request or response globally – add auth headers, log requests, implement retry logic, all without touching individual calls.

const axios = require('axios');
// Add an auth header to every outgoing request
axios.interceptors.request.use((config) => {
  config.headers['Authorization'] = `Bearer ${process.env.API_TOKEN}`;
  return config;
});
// Log every failed response
axios.interceptors.response.use(
  (response) => response,
  (error) => {
    if (error.response) {
      console.error(
        `[${error.response.status}] ${error.config.method.toUpperCase()} ${error.config.url}`
      );
    }
    return Promise.reject(error);
  }
);

For scraping, a response interceptor that retries on 429 (rate limit) or 503 (server overload) with exponential backoff is practically essential. You wire it up once and forget about it.

Creating a reusable instance

When all your requests share the same base URL, auth headers, and timeout, create a configured instance with axios.create() instead of repeating the same config everywhere.

const axios = require('axios');
const api = axios.create({
  baseURL: 'https://api.github.com',
  timeout: 10000,
  headers: {
    'Authorization': `Bearer ${process.env.GITHUB_TOKEN}`,
    'Accept': 'application/vnd.github.v3+json',
    'User-Agent': 'my-node-script/1.0',
  },
});
// Now every call uses the shared config
const user = await api.get('/user');
const repos = await api.get('/user/repos');

This also keeps connection pools alive between requests, which matters when you're hitting the same API hundreds of times. Create the instance once, reuse it everywhere.

Translating cURL commands to JavaScript

This is the section you'll probably need to bookmark. You have a cURL command, and you need it in JavaScript. Rather than guessing, here's a systematic way to translate any cURL command flag by flag.

The "copy as cURL" workflow

Before you translate anything, you need the cURL command. If you're trying to replicate a request your browser made, Chrome and Firefox hand it to you for free:

  1. Open DevTools (F12 or Ctrl+Shift+I).
  2. Go to the Network tab.
  3. Find the request you want to replicate.
  4. Right-click it and select Copy, then Copy as cURL.

That gives you the exact command as a cURL string. This is the fastest way to get a working starting point, especially when you're debugging why your JavaScript request behaves differently from what the browser sent.

Flag-by-flag mapping

Here's how each common cURL flag translates to Fetch and Axios. If you've been following along with the earlier sections, most of these will look familiar.

cURL flag

What it does

Fetch equivalent

Axios equivalent

-X METHOD

Sets the HTTP method

method: "METHOD"

method: "METHOD"

-H "Key: Value"

Adds a header

headers: { "Key": "Value" }

headers: { "Key": "Value" }

-d '{"key":"value"}'

Sends a request body

body: JSON.stringify({key: "value"})

data: { key: "value" }

--data-urlencode

Sends URL-encoded data

body: new URLSearchParams({key: "value"})

data: new URLSearchParams({key: "value"})

-u user:pass

headers: { "Authorization": "Basic " + btoa("user:pass") }

auth: { username: "user", password: "pass" }

--compressed

Accepts gzip/deflate

Automatic, no action needed

Automatic, no action needed

-x host:port

Routes through a proxy

Custom agent (see proxy section)

httpAgent / httpsAgent with proxy agent

-k / --insecure

Skips TLS verification

agent: new https.Agent({ rejectUnauthorized: false })

httpsAgent: new https.Agent({ rejectUnauthorized: false })

-L / --location

Follows redirects

On by default (disable with redirect: "manual")

On by default (disable with maxRedirects: 0)

-o filename

Saves response to a file

fs.writeFile() after fetching as ArrayBuffer

fs.writeFile() with responseType: "arraybuffer"

Full example: translating a real cURL command

Let's take a GitHub API request that creates an issue – the kind of command you'd realistically copy from documentation or DevTools.

The cURL version:

curl -X POST https://api.github.com/repos/octocat/hello-world/issues \
  -"Authorization: Bearer ghp_your_token_here" \
  -"Accept: application/vnd.github.v3+json" \
  -"User-Agent: my-script/1.0" \
  -'{"title": "Bug report", "body": "Something is broken", "labels": ["bug"]}'

Translated to Fetch:

async function createIssue() {
  const response = await fetch(
    'https://api.github.com/repos/octocat/hello-world/issues',
    {
      method: 'POST',
      headers: {
        'Authorization': 'Bearer ghp_your_token_here',
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'my-script/1.0',
      },
      body: JSON.stringify({
        title: 'Bug report',
        body: 'Something is broken',
        labels: ['bug'],
      }),
    }
  );
  if (!response.ok) {
    throw new Error(`GitHub API returned ${response.status}`);
  }
  const issue = await response.json();
  console.log('Created issue:', issue.html_url);
}

Translated to Axios:

const axios = require('axios');
async function createIssue() {
  const response = await axios.post(
    'https://api.github.com/repos/octocat/hello-world/issues',
    {
      title: 'Bug report',
      body: 'Something is broken',
      labels: ['bug'],
    },
    {
      headers: {
        'Authorization': 'Bearer ghp_your_token_here',
        'Accept': 'application/vnd.github.v3+json',
        'User-Agent': 'my-script/1.0',
      },
    }
  );
  console.log('Created issue:', response.data.html_url);
}

Same result, different trade-offs. The Fetch version is more explicit, as you see every step. The Axios version is shorter because it handles JSON serialization and POST body formatting automatically. Neither is wrong; it depends on what your project already uses and how much boilerplate you're willing to manage.

Translating basic auth

cURL's -u flag trips people up because there's no direct one-to-one option in Fetch. You need to construct the Authorization header with a Base64-encoded string manually.

cURL:

curl -u admin:secretpass https://api.example.com/protected

Fetch:

const response = await fetch('https://api.example.com/protected', {
  headers: {
    'Authorization': 'Basic ' + btoa('admin:secretpass'),
  },
});

Meanwhile, Axios has a dedicated auth option that does this for you:

const response = await axios.get('https://api.example.com/protected', {
  auth: {
    username: 'admin',
    password: 'secretpass',
  },
});

Downloading files

cURL's -o flag saves the response directly to a file. In JavaScript, you fetch the response as binary data and write it yourself.

cURL:

curl -o report.pdf https://example.com/files/report.pdf

Fetch (Node.js):

const fs = require('fs');
async function downloadFile() {
  const response = await fetch('https://example.com/files/report.pdf');
  const buffer = Buffer.from(await response.arrayBuffer());
  fs.writeFileSync('report.pdf', buffer);
  console.log('Downloaded report.pdf');
}

Axios:

const axios = require('axios');
const fs = require('fs');
async function downloadFile() {
  const response = await axios.get('https://example.com/files/report.pdf', {
    responseType: 'arraybuffer',
  });
  fs.writeFileSync('report.pdf', response.data);
  console.log('Downloaded [report.pdf](/blog/curl-download-files)');
}

The key detail is responseType: 'arraybuffer' in Axios. Without it, Axios tries to parse the response as JSON, and you get garbage.

Online converters

If you'd rather skip the manual translation, curlconverter.com takes a pasted cURL command and generates Fetch, Axios, or plain Node.js code automatically. It's useful for long, complex commands where counting flags and quotes by hand is tedious.

That said, understanding the mapping yourself is more practical in the long run. Converters are great for one-off translations. When you're iterating on a scraper or debugging a failing request, knowing which flag maps to which option means you can fix things in seconds instead of switching between tabs.

Adding proxy support for web scraping

A scraper that works locally can still run into rate limits, IP blocks, or location-specific content when used at scale. Proxies help by routing requests through different IP addresses, making it possible to rotate IPs and send requests from specific locations.

Why proxies matter for JavaScript scrapers

let you distribute requests across multiple IP addresses instead of repeatedly connecting from the same one. Rotating proxies let you distribute requests across multiple IP addresses instead of repeatedly connecting from the same one.

cURL proxy integration

With cURL, use the -x flag to route a request through a proxy:

curl -x http://username:password@gate.decodo.com:7000 https://ip.decodo.com/ip

You can also configure a proxy through the HTTPS_PROXY environment variable:

export HTTPS_PROXY="http://user:pass@gate.decodo.com:7000"
curl https://ip.decodo.com/ip

The same principle applies in Node.js, although the exact proxy configuration depends on the HTTP client you use.

Proxy setup in Axios

Axios can route Node.js requests through a proxy agent. For a complete setup with examples and configuration options, see our Axios proxy guide.

A typical setup uses https-proxy-agent and passes the resulting agent through Axios's httpsAgent or httpAgent option.

Proxy setup in Fetch

Proxy configuration with Fetch depends on the implementation. Node.js's Undici-based Fetch, for example, supports a ProxyAgent through the dispatcher option.

For detailed implementations, see our Node Fetch proxy guide. If you're building a scraper around Fetch, the Node Fetch web scraping guide covers the broader workflow.

Residential vs. datacenter proxies

The right proxy type depends on the target and scale of your scraper. Datacenter proxies are generally fast and cost-effective, but their IP ranges can be easier for websites to identify as non-residential traffic.

Residential proxies use IP addresses associated with residential internet connections, which can make them better suited to targets with stricter anti-bot systems or location-specific content.

For demanding scraping tasks, residential proxies are often the more reliable option, while datacenter proxies can work well when speed and cost are the priority.

Choosing the right approach

Pick based on your environment and constraints. Here's a basic breakdown:

Approach

Works in browser

Works in Node.js

Dependencies

Proxy support

Recommended for

Fetch (native)

Yes

Yes (18+)

None

Manual setup

Simple API calls, lightweight scripts

Axios

Yes

Yes

1 package

Via agent

Default choice for scraping and APIs

child_process + cURL

No

Yes

None

Native cURL flags

Reusing existing cURL commands

node-libcurl

No

Yes

Native addon

Built-in

Advanced networking control

Web Scraping API

Yes

Yes

API call

Handled for you

Anti-bot, CAPTCHA, JS-heavy sites

General takeaway:

  • Default to Axios for most Node.js scraping work
  • Use Fetch when you want minimalism, or when you're in the browser
  • Treat child_process and node-libcurl as edge cases, not starting points
  • If you're fighting rate limits, CAPTCHAs, or fingerprinting, the HTTP client isn't the bottleneck anymore. Offload it to a scraping solution

Best practices and common mistakes

These are the things that quietly break scraping scripts in production. Fix them up front, and you avoid hours of debugging later.

  • Set request timeouts. Configure a reasonable timeout so stalled requests don't block your process.
  • Use a realistic User-Agent. Default headers like axios/1.x.x can make automated traffic easier to identify, so set an appropriate User-Agent for your requests.
  • Handle HTTP errors. Check response.ok with Fetch and log relevant status codes and response details when requests fail.
  • Don't hardcode credentials. Keep API keys, tokens, and proxy credentials in environment variables rather than your source code.
  • Implement retries with exponential backoff. Use limited retries with exponential backoff for transient errors such as 429 and 503 responses.
  • Reuse Axios instances. Create shared instances with axios.create() to reuse configuration and connections across requests.
  • Remember browser CORS restrictions. Browser-side requests to other domains may be blocked by CORS.
  • Don't overbuild around blocked requests. If retries and request tweaks aren't solving the problem, consider a scraping API or proxy-backed solution instead of adding more workarounds.

The pattern is simple: control timeouts, handle failures, and avoid leaking state. Do that consistently, and your HTTP layer stops being the problem.

Final thoughts

You have four ways to bring cURL-style requests into JavaScript – child_process for running raw cURL commands, node-libcurl for low-level control, Fetch for a built-in and dependency-free option, and Axios for a cleaner, production-friendly experience. In practice, most API and scraping work is covered by Fetch or Axios, with Axios being the default when you want less boilerplate and better error handling. The other two exist for edge cases, not everyday use. And once you start dealing with heavy bot protection, retries, and proxy rotation, the HTTP client stops being the interesting part – that is where handing things off to a scraping API makes more sense than building around limitations.

Scraping shouldn't be this hard

Replace proxy configs, retry logic, and fingerprint workarounds with a single API call that returns clean data.

Share article:

About the author

Zilvinas Tamulis

Technical Copywriter

A technical writer with over 4 years of experience, Žilvinas blends his studies in Multimedia & Computer Design with practical expertise in creating user manuals, guides, and technical documentation. His work includes developing web projects used by hundreds daily, drawing from hands-on experience with JavaScript, PHP, and Python.

Connect with Žilvinas via LinkedIn

All information on Decodo Blog is provided on an as is basis and for informational purposes only. We make no representation and disclaim all liability with respect to your use of any information contained on Decodo Blog or any third-party websites that may belinked therein.

Frequently asked questions

Can I use cURL directly in JavaScript?

Yes. In Node.js, you can run cURL through the built-in child_process module, but cURL must be installed on the system and its output needs to be handled by your script. For most projects, Fetch or Axios is a simpler choice.

What is the JavaScript equivalent of cURL?

The closest built-in equivalent is the Fetch API, available in modern browsers and Node.js 18+. Axios is a popular alternative that adds conveniences such as automatic JSON parsing, built-in timeouts, and automatic errors for unsuccessful HTTP status codes.

Is Axios better than Fetch for web scraping in Node.js?

Both work well for web scraping. Axios provides conveniences such as automatic JSON parsing, built-in timeouts, and interceptors, while Fetch is built into Node.js 18+ and requires no additional package. The better choice depends on how much request handling you want to configure yourself.

How do I add a proxy to a JavaScript HTTP request?

Proxy setup depends on the HTTP client. Axios can use a proxy configuration or proxy agent, while Node.js Fetch can work with Undici's ProxyAgent through the dispatcher option. For more demanding scraping tasks, a rotating residential proxy can distribute requests across multiple IP addresses.

How do I convert a cURL command to JavaScript?

Break the cURL command down by its flags: -X maps to the request method, -H to headers, and -d to the request body or Axios data. You can then recreate the request using Fetch or Axios, following the same method, headers, body, and authentication settings.

JSON response panel showing scraped product data, with 'eCommerce store' preview and 'Scraping' label on dark background.

How to Do Web Scraping with curl: Full Tutorial

Web scraping is a great way to automate the extraction of data from websites, and curl is one of the simplest tools to get started with. This command-line utility lets you fetch web pages, send requests, and handle responses without writing complex code. It's lightweight, pre-installed on most systems, and perfect for quick scraping tasks. Let's dive into everything you need to know.

JS logo overlaying a glowing blue code snippet on a dark abstract background

JavaScript Web Scraping Tutorial (2026)

Ever wished you could make the web work for you? JavaScript web scraping allows you to gather valuable information from websites in an automated way, unlocking insights that would be difficult to collect manually. In this guide, you'll learn the key tools, techniques, and best practices to scrape data efficiently, whether you're a beginner or a developer looking to streamline data collection.

Two neon-green code windows connected by dashed lines representing data flow on a dark background

Web Scraping with Cheerio and Node.js: A Comprehensive Guide

Scraping static web pages can be challenging, but Cheerio makes it fast and efficient. Cheerio is a lightweight Node.js library that parses and manipulates HTML using a syntax similar to jQuery. This guide covers key concepts, practical code examples, and essential techniques to help you extract web data with ease—no matter your experience level.

© 2018-2026 decodo.com (formerly smartproxy.com). All Rights Reserved