top | item 18394698

Show HN: A Node.js stream library that works with promises

10 points| reconbot | 7 years ago |npmjs.com

3 comments

order

reconbot|7 years ago

Hi I built this little stream processing library so I could write things like;

  const stats = await pipe(
    Nodes.scan({ fields: true }),
    map(generateStats),
    tap(() => count++),
    reduce(mergeGraphStats, {})
  )

and have generateStats and mergeGraphStats be async functions and not have to worry about error handling and pushing more than one object at a time in a read stream. We use it to process billions of events and objects a day. It makes nodejs streams fun to use.

Hope you find it as useful as we do.

oliverx0|7 years ago

Pretty cool. Could the following be used as a baseline to create the same functionality?

    async function * nextStreamEntry() { 
      while(true){
        const entry = yield;
        const result = await processEntry(entry);
      }
    }

    // Initialize the iterator
    const iter = nextStreamEntry();
    iter.next();

    // For every entry of the stream
    for (const monster of results) {
      iter.next(monster)
    }