Yml Coding

[xiaq]

Saying this is reinventing BASIC in YAML seems to miss the point here; the programming model here is essentially a finite state machine with optional data flowing between nodes, and AFAIK there isn't really a widespread language that targets this model.

And a restricted model is not there just so that non-programmers can use it. Restricting what you can do in the program means that there are more things you can do with the program. I haven't checked if GCP's Workflows supports everything below, but here are some things you can do in principle:

* You can visualize the entire program as a flowchart, and visualize the state of the program simply by pointing at a node in the flowchart. This is not possible with a general purpose language since there could be arbitrary levels of call frames.

* You can implement retry policies for each step entirely transparently, and possibly other things like authentication. Aspect-oriented programming is more practical when the programming model is restricted.

* You can schedule the steps onto different hosts, possibly in parallel.

* You can suspend and resume the workflow, since its entire state is just which step is being executed, plus a handful of variables (which presumably are always serializable).

Re the problem of extension: the idea seems to be that you put all "smartness" inside HTTP services that are written in real languages and only use this as a dumb glue language.

[mfateev]

Look at my project temporal.io. It does the same thing using general purpose programming languages. Currently Java and Go are supported with Python, Ruby and C# under development. This is absolutely valid workflow code:

     public void execute(String customerId) {
       activities.onboardToFreeTrial(customerId);
       try {
         Workflow.sleep(Duration.ofDays(60));
         activities.upgradeFromTrialToPaid(customerId);
         while (true) {
           Workflow.sleep(Duration.ofDays(30));
           activities.chargeMonthlyFee(customerId);
         }
       } catch (CancellationException e) {
         activities.processSubscriptionCancellation(customerId);
       }
    }

[roenxi]

That is good, but why code all that up in YAML? This looks to me like a programming language with great efforts made to obfuscate the fact.

We've had 70 years to figure out what good programming language layout looks like. I think there is almost a consensus that it doesn't look like the linked code-wearing-a-fake-moustache .

Are people supposed to code these state machines using some sort of real language then translate it to YAML?

[lowercase1]

This is similar to AWS step functions which kinda descended from more general/powerful AWS Simple Workflow.

[indymike]

Actually, it isn't even as capable as BASIC, but it is very cool and probably really useful.

[xiaq]

A belated disclaimer: I work for GCP, but not this product.

[bonestormii_]

Data is data. Code is logic. So the the thought process goes...

...what if I could store my configuration (data! right?!) in a way that is nicely separated from the logic?! No more scripting for me!

Oh wait. Some of the configuration can't be generalized about universally. Configurations fundamentally contain logic, I guess.

Oh, well then why don't I just represent the logic in the data?! That will be much better than representing the data in the logic!

....but now, you are back where you started, only instead of using something nice, standard, and powerful like Python, you have to use this... language... thing. This YAML convention you cooked up.

Separate the data from logic. Read the data into the logic. Make a nice organized place to call custom logic from... like, you know, a file directory full of scripts, which are called according to some scheme. It could be another data file. Stop there.

Like this:

    $ ls -Ra
    ./config/do_something.yaml
    ./config/do_something_else.yaml
    ./config/config.yaml
    ./logs/ping.log
    ./scripts/do_something.py
    ./scripts/do_something_else.py

---

    $ cat ./config/config.yaml

    do_something:
        target: my.stupid.server
        exec_frequency: daily
        ...
    do_something_else:
        target: my.stupid.server
        exec_frequency: monthly
        ...

---

    $ cat ./config/do_something.yaml

    ping: true
    output: 
       directory: ./logs/ping.log
       mode: append

---

    $ cat ./config/do_something_else.yaml

    delete: ./logs/ping.log

---

    $ cat ./scripts/do_something.py

    from stupid.library.task import Task, subscribe

    class DoSomething(Task):

        @subscribe
        def ping(self, target):
            return super().ping(target)

Then you code the program that runs collects all of the task methods, put them in an ordered list, and run them if the conditions in the config.yaml is met.

Every other thing is done in a task method in python, or something like it. I don't think we need more abstraction than that.

[pydry]

This is more or less the structure I followed with this testing framework: https://github.com/hitchdev/hitchstory

The whole idea being that you don't want the story to be turing complete (there are no loops or conditionals with a story), but the code that executes it needs to be turing complete.

[abeppu]

I get the appeal of something like this. It's locked down enough that it's perhaps challenging to do something really unsafe with it. It's maybe usable by someone who doesn't feel comfortable with a "real" programming language.

But it seems like this falls apart as soon as one service you need to interact with creates a requirement not anticipated by this very constrained tool-set. You need to query service A, extract something with a regex, base64 encode something else before you post to service B? Well we didn't include regexes, a module/import system, or the ability to introduce UDFs in a different language.

And if you had the resources to make all your services play into the expectations of this workflow system, you might not need to use this workflow system.

[rightbyte]

I don't see how writing a kinda abstract syntax tree with the opacity of Enterprise JavaBeans is easier than say writing the workflow in some Basic dialect with only GOTOs and IFs.

[perfunctory]

And here we come. Google, in the 21st century, is trying to sell us BASIC

  - define:
    assign:
        - array: ["foo", "ba", "r"]
        - result: ""
        - i: 0
  - check_condition:
    switch:
        - condition: ${len(array) > i}
            next: iterate
    next: exit_loop
  - iterate:
    assign:
        - result: ${result + array[i]}
        - i: ${i+1}
    next: check_condition
  - exit_loop:
    return:
        concat_result: ${result}

edit: it's not April 1st yet, is it?

[baq]

i've seen this before, as has my father and probably my grandfather, except it was thought out better and was called lisp instead of yaml.

[1337shadow]

I fail to see how better it is than:

    array = ["foo", "ba", "r"]
    result = ""
    i = 0
    while len(array) > i:
        result = result + array[i]
        i += 1

exactly 3 times less code

[qppo]

Well at least it's NoCode

[recuter]

Come on, this is bound to revolutionise your Kubernetes configuration.

[tuankiet65]

Trying to turn YAML into a programming language reminds me of Ansible playbooks.

[oblio]

http://ant-contrib.sourceforge.net/tasks/tasks/if.html

[dhosek]

We went through great lengths to be able to do parameterized YAML using jinja2 templates. I designed a system that let us effectively create a new yaml out of two other yamls which let us handle repetitive configuration tasks reasonably well but there were occasional unexpected challenges thanks to the syntactically significant white space of yaml.

[osmarks]

This seems like some sort of horrible inner platform effect. It would be nice if they would use an actual embeddable programming language, like Lua.

[baq]

turing complete configuration? no thanks. i like it when my configs are guaranteed to halt.

[amyjess]

Lua isn't homoiconic, though.

This is basically Lisp without the parentheses.

[swiley]

IMO: most of these weird DSLs come from wanting a data structure but getting confused and writing a bad programming language instead.

[thangalin]

Tangentially related is my YAML pre-processor that performs string interpolation:

https://bitbucket.org/djarvis/yamlp/

Along with Pandoc, it allows common prose from Markdown to be de-duplicated, as described in my Typesetting Markdown series:

https://dave.autonoma.ca/blog/2019/07/06/typesetting-markdow...

Obviously it's laborious insert to YAML keys everywhere, so I'm developing an editor that integrates YAML and plain text document formats (such as Markdown):

https://www.youtube.com/watch?v=u_dFd6UhdV8

[jpxw]

Even writing CI configuration is onerous enough in YAML, let alone a program

[orf]

I like this. Go "har har har yaml" all you like, but this reads a damn lot better than AWS step functions and seems way more powerful.

There are a number of tasks that benefit from something like this, and there are a huge number of advantages from being able to encode steps in domain specific languages such as YAML and have them execute with truly "no code".

[osmarks]

This is not "no code". This is code, imperative code even, in a really bizarre format.

[baq]

yaml is not a language. it's a serialization format.

this thing here is a language that serializes into yaml. there are zero benefits of using it except perhaps 'it is yaml so i don't have to compile my configuration'.

note that xml with xslt would have handle it better.

[jpalomaki]

Do they have a graphical tool for defining these workflows? For me this looks like the business process management tools from the past. With those the idea was not to use them to write program logic, but connect the dots on very high level. See for example the use case pictures at [1].

[1] https://cloud.google.com/workflows

[cordite]

How about we use resumable lua scripting

[dpc_pw]

Peak Yaml?

[throwaway4889]

Yaml is the new xml

[fmakunbound]

Good god it’s stuff like this makes me contemplate leaving professional programming.

[wickedOne]

despite the readability of the yaml syntax, for me the big disadvantage is the inability to use meta / type info.

for a straightforward collection of strings: probably good enough, for anything else xml is more likely to cover the usecase better

[Spivak]

YAML has really good typing support with type annotations being a first class citizen. But you’re dependent on the hosting software (adding or understanding) the types.

For example

    start: !date 2020-09-12

could actually map to a native date object.

[taylorlapeyre]

I am surprised at how well this actually reads!