top | item 33817557

(no title)

mothcamp | 3 years ago

You could start by looking into either multitask transformers or really general seq2seq models like T5. With T5, for example, it just learns to transform one text sequence into another. So you could fine-tune T5 to produce your target sequence, but rather than outputting an explicit Python list of tuples, it would output a string that looks like a sequence of tuples.

Or maybe skip all that and outsource it to GPT: https://imgur.com/a/BQv6C3K

discuss

order

brooksbp|3 years ago

Ah, so if the model is just converting input text into output text, it can really learn how to do just about anything? But, there may be certain aspects of model design that make it better at some types of conversions ("tasks") than others? And there may be certain data sets that you want to train a base model on to get base learning of such as general language comprehension, and then build on top of that for your specific use case?

mothcamp|3 years ago

Yeah, I can see that being the case for specialized domains. With state-of-the-art models widely available to the public, knowledge of the domain and its workflows, and fine-tuning models to suit the domain will probably be your edge.

trenchgun|3 years ago

It is kind of like a very opaque but trainable Turing machine.