[noctarius]

I guess your biggest issue may be the multiple writer problem, but you'd have the same issue on a local disk. The second multiple writer are supposed to update the same files, you'll run into issues.

Have you thought about TCP sockets between the apps and sharing state, or something like a redis database?

[willcodeforfoo]

Hmm, maybe... although that wouldn't help with the aggregating multiple nodes issue, or would require a lot of app-side logic.

In this example, I have 200GB of ephemeral storage available on each node, ideally I'd like something like this:

  node1: /tmp/data1 (200GB free space)
  node2: /tmp/data2 (200GB free space)
  node3: /tmp/data3 (200GB free space)
  node4: /tmp/data4 (200GB free space)
  node5: /tmp/data5 (200GB free space)

...pods could some how mount node{1..5} as a volume, which would have 5 * 200GB ~1TB of space to write to... multiple pods could mount it and read the same data.