Dataset & RAG Module
Dataset module will consists of datasets/files that can be used within prompts.
Dataset can be in the form of txt, pdf and etc.
https://hooga-gaming.atlassian.net/wiki/spaces/Product/pages/155811853/Weave+-+PRD+-+RAG+Doc+Processing+and+Dataset+Module
Log in to comment and vote
Comments3
Mei Wei Lim
Dec 5, 2023
support any text based files (txt, csv, json) & pdf
Mei Wei Lim
Dec 5, 2023
https://chasm-talk.slack.com/archives/C04AGUJE5LH/p1701742847285399
Mei Wei Lim
Dec 5, 2023
About the PDF reading ticket, the backend team should implement a return function for instances when a PDF cannot be read. This will enable the frontend to specifically handle such PDFs, informing the user that the PDF is unreadable and prompting them to either ignore it or upload a different version.Suggested flow:
The front-end loads the PDF.
The PDF is submitted to the backend for processing.
If the PDF is valid and processing is successful:
The backend returns a preview to the front-end.
The user then clicks to run the process.
If the PDF is invalid or processing fails:
The backend notifies the front-end that the PDF cannot be processed.
The front-end displays a message to the user indicating the PDF is unreadable and provides options to reupload or ignore.
Depending on the user’s choice, the process either repeats from step 2 or moves to step 3 if a new, valid PDF is uploaded.