Fetch API has landed into Node.js

[inglor]

Hey, Node core person here (and the person who triggered the land) - we're super excited for this and would love help and feedback.

This is still experimental and we'd love to hear from the community what you'd like to see.

[zebracanevra]

Is there any chance to break spec and allow manual redirect handling? the fetch API makes a lot of sense in a browser, but imo this is a pretty crucial feature that undici's implementation lacks. [0]

Deno decided to break spec [1][2] so the following code works fine:

    fetch('https://httpbin.org/status/302', {redirect: 'manual'})
      .then(res => console.log(res.status, res.headers))

In undici this will succeed but with res.status 0 and no headers, as per spec. You aren't allowed to see the content of a redirect, like in a browser.

[0] https://github.com/nodejs/undici/issues/1072 [1] https://github.com/denoland/deno/pull/8353 [2] https://github.com/denoland/deno/issues/4389

[asdfman123]

Thank you for making "fetch" happen

[ignoramous]

Hi there,

> ...we'd love to hear from the community what you'd like to see.

In our tests, fetch on Deno was wayy faster than undici fetch that now has been merged into nodejs.

I couldn't figure out why that was case, but forwarding requests over node's http2 client (instead of undici fetch) then had comparable (but not as fast) performance as Deno (presumably because our hand rolled impl lacked connection pooling).

[franciscop]

Really happy to see fetch() merged into core! Looking at the PR, what are those WASM blobs in Undici and how is it that Node.js accepts a random very large compiled blob instead of asking for the source file + compilation step? https://github.com/nodejs/node/commit/6ec225392675c92b102d3c...

[nefitty]

Hey this one change is going to positively impact my day to day life. Just wanted to let you know. Thanks for sharing your hard work and time.

[staticelf]

Quick question, judging from the commit alone this seems like a rather small change. I assume there's more to it though. I just wonder how come features like this that kind of seems obvious to include in the ecosystem takes quite some time to land? I understand the reality is more complex perhaps, so I am genuinely curious.

I hope you realize no disrespect, this will greatly improve the daily work for me since I use node at work every day.

[neves]

Where can I find an explanation about what is Fetch and why it should be in Node.js?

[kichimi]

I do have a question that's only tangibly related to fetch, I was wondering what Nodes stance is on the seemingly increasing gap between the NodeJS and browser implementations of the JavaScript engine.

It's still very common to see Node modules that use require(), for example, but would otherwise work flawlessly in a browser where there is no require() support (without using shims and other libraries).

[mitchellgoffpc]

Huge thanks to the node team for adding this, I’ve been wanting fetch in node for years now! Installing node-fetch for every project was getting kind of old haha

[adityapatadia]

Thanks a ton for this. Fetch API is elegant and it working on node natively will so many edge cases we need to handle currently. Thanks a ton, please know that this will positively change our life and tens of thousands of other developers worldwide.

[can16358p]

Great work!

Curious to check but quick question: does it support upload progress? If not, is it considered?

[tbarbugli]

yes, yes, yes.

This was a huge pain for us. We develop an SDK that needs to work on Node, browsers and React Native.

I am super happy to hear about this :)

[joshxyz]

Does it mean we have a built-in FormData in Node.js core for multi-part requests?

[ivanb]

Is there support for timeouts? It's the main reason I use https://github.com/sindresorhus/got

[mnutt]

Do you have any sense of the expected performance of web streams as compared to the existing node.js streams?

[gk1256]

I wish if Node had a way to enforce any case in its requests.

[rglover]

This is awesome, thank you. Excited to take it for a spin.

[andrew_]

pretty great. I've been using undici for a while now and very happy with the speed and experience.

[eurasiantiger]

FormData and Blob/File support?

[pseudosavant]

Every Node.js dev should be thanking the Deno creators. It has forced them off of their "this is how Node does it" complacency. Most improvements I notice and care about on Node these days are all things that Deno already supports (fetch, ES modules, async std lib uses promises, etc).

The Web Platform APIs aren't perfect, but they are better, and more thoroughly thought out than Node's.

[inglor]

What is "this is how Node does it" complacency you speak of?

Node has been talking about Fetch since at least 2018 _before_ Ryan even announced Deno. This is true for promises, ESModules etc. Ryan _attended those meetings_ in the Node collaborator summit.

Don't get me wrong I am thankful for Deno (and a contributor!) but:

- It's a lot easier to make changes when the project is new and you don't have a lot of users (which is great for Deno) whereas Node has a whole (huge) ecosystem to consider on every change. - It's a lot easier to make changes with venture capital and funding to hire a bunch of people to work on it full time vs. a project with a "regular" MIT license where the code is owned by the community and not a company.

Those are not complaints about Deno, I like Deno (and not just Ryan, Lucas, Kit, Ben and the other people are nice and helpful and the project is really nice).

It's just to explain very little of Node's innovation is (at the moment) driven by Deno.

[thyrox]

My biggest issue with node when I was working on it briefly was I couldn't do the 'import' statements like wepback. Are they supported too now?

I'm not a web developer so I had a very hard time understanding why there are so many different type of imports in JavaScript like require, import, umd, amd, etc and which one works in browser and which one works in node?

Also why do so many libraries have this strange 5 line header code for this umd, amd, business. Is that to make their packages work with nodejs?

Does anyone who knows enough JavaScript point me in the right direction about it. I find all this very confusing.

[ezekg]

One thing I hate about fetch() is that you can't manually follow redirects. Using { redirect: 'manual' } doesn't expose the Location header of the redirect, so it's essentially useless. I know that node-fetch fixed this issue, so I hope the official Node fetch() does not have the same problem.

[nfriedly]

I love fetch, except for the way that it combines headers. If more than one of the same header is set (such as multiple set-cookie headers), it combines them all into a single comma-separated string. I know this is allowed in the HTTP spec, and it's probably even a sensible default. But the fetch spec doesn't allow any access to the raw headers, so there's no straightforward way to get the original uncombined headers back. Set-Cookie headers, in particular, often contain commas in their expiration dates, so just splitting around `, ` will lead to problems.

I took a look through the source of this new fetch API, and it seems to have inherited that wart: https://github.com/nodejs/undici/blob/2dd3437e20c5a3cc466226...

I've argued with the authors of the fetch spec about this before, and ultimately settled on using https://www.npmjs.com/package/set-cookie-parser#user-content... to work around this flaw. (For clarity, I published the package, but chrusart wrote that method - https://github.com/nfriedly/set-cookie-parser/pull/19)

[styfle]

There's a relevant spec issue here https://github.com/whatwg/fetch/issues/973

[jitl]

There’s an NPM package containing this implementation called ‘undici’ that you can install in previous versions of Node; the package.json claims support back to 12.x.

https://www.npmjs.com/package/undici

However the fetch implementation itself is documented as unstable and only supports Node 16.x. I believe this is because it is not quite spec compliant yet, so there is latitude for breaking changes that make it more spec compliant.

[krossitalk]

As someone browser focused this surprised me. I've been using fetch() for years now.

[easton]

For those of us unfamiliar, how long until stuff comes out of experimental?

[esamatti]

Does this contain `request.formData()` and `FormData`?

https://developer.mozilla.org/en-US/docs/Web/API/Request/for... https://developer.mozilla.org/en-US/docs/Web/API/FormData

[inglor]

> `request.formData()`

Yes.

> FormData

Yes, but not in that PR since `FormData` needs to behave differently as part of the platform but there is intent to support it before moving from experimental.

[stevage]

Out of curiosity, what took so long? It's a small API that has been stable for a long time? I'm probably missing something.

[chx]

Good for NodeJS

However, I feel the fetch API is completely botched because it lacks timeout support. I have cobbled some together for https://github.com/Backblaze/gists/pull/8 but gosh. I really hope all that is actually not necessary :/

[Meic]

Undici seems to be an HTTP 1.1 client. Is there any work or timescale to provide HTTP 2 support?

[bricss]

check https://www.npmjs.com/package/rekwest

[songzme]

sorry if this is a stupid question, but how do I try this out? I ran node with the experimental flag on a file that calls fetch:

node server.js --experimental-fetch

Got 'fetch is not defined' reference error.

[inglor]

First: not a stupid question - the release cadence, multiple release lines and other minutia of a project like Node are not something I'd expect users to be intimately familiar with!

This _just landed today_. You can get it by building from master:

```

# there is more details in building.md

git clone https://github.com/nodejs/node

cd node

# may need to pass --openssl-no-asm

./configure

make -j12

./out/Release/node --experimental-fetch

```

Otherwise - wait a bit for the next v17 release to land per the normal release cycle :)

[sealthedeal]

Ive used Fetch in the past, great lib. I'm curious why Node core has decided to make this an official part of Node? From one perspective there could be a hard argument that this should never be a part of Node as it is an external library and not a building block of the language? Almost feels like feature creep for the core language? I know that's not the intent, but wanting to understand the logic behind this decision :)

[ramblerman]

fetch is part of the client side standard: https://fetch.spec.whatwg.org/, and had been for a while.

I don't think it is so strange for node to incorporate it as part of their standard lib as well.

1. https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API

[sealthedeal]

FWIW, this was the answer I was hoping to get, grabbed from a separate comment :)

"With fetch available in node, we have a single, standardised, promise-based HTTP client Interface available on all (more or less common) JavaScript platforms.

This is great news for an ecosystem traditionally extremely fragmented and may lead the way to more consolidation in the JS library space. "

[wruza]

I second this, since fetch is a wrapper around xhr (though doesn’t have to be because node is a non-restricted runtime), and it diverged already for redirect semantics. It’s news, but is it big?

[prollings]

They're talking about JavaScript's fetch API. It's not an external library.

[pitaj]

Fetch is a standard browser API.

[mcraiha]

Why it took so long? e.g. Deno had fetch support for ages

[bachmeier]

Good that some people are happy about this, but it really is pointless to post a link to "lib: add fetch" followed by code diffs. HN should have a requirement that there be an actual post, rather than a discussion starter for those in a particular community. All you'd have to do is type in a brief text post with an explanation and then link to the code diffs for those interested.

[inglor]

There will be a blog post on this in the Node.js blog and official media communication. This _just_ landed today and I suspect the HN crowd are more in the "get news early" camp and not "wait for the official press release" camp.

[outside1234]

Thank god - no more node-fetch shims for node.js!

[quickthrower2]

Funny how my brain works, I thought it was there already, but I must have got so used to having it as a dependency I forgot that it was a dependency not built in. To be fair I don't "Node" all the time! It's on and off.

[adam_arthur]

Awesome work! This was one of the main remaining pain points I had with node.

[siquick]

Any reason to use fetch if I’ve been happy with axios for years?