You don’t. You cut it into snippets. For those you create embeddings which allow you to rank them by semantic similarity to a query. You then prompt GPT3 with the question plus, say, the three most relevant snippets from the book.
The most difficult thing about the process is preventing the model from making stuff up.
This is exactly what I'm working on! My project is taking Zoom conversation, using pyannote for speaker diarisation, whisper for transcription, pinecone.io for semantic search, then feeding that into GPT-3 so we can ask questions about conversation.
For us this is super useful because it's not unusual for our discover sessions to last days and we're all terrible at taking notes.
As a nerd, my brain is already buzzing on ways that I could use this for my groups D&D campaigns.
leobg|3 years ago
The most difficult thing about the process is preventing the model from making stuff up.
kreas|3 years ago
For us this is super useful because it's not unusual for our discover sessions to last days and we're all terrible at taking notes.
As a nerd, my brain is already buzzing on ways that I could use this for my groups D&D campaigns.
d4rkp4ttern|3 years ago