Web Scraping

Most AI's (like ChatGPT and Bard) are trained on data from the internet. This data has a cut off date of when it was collected to train the model. For instance, the cutoff date of ChatGPT 3.5 is September 2021.

So, if you ask it a question about the latest news article or current events, it just doesn't know since it has no knowledge of it.

This is where web scraping comes in. We've built in the ability for prompts to utilize web scraping by making requests to our services before the prompt is sent to the AI (ChatGPT, Bard, Claude, etc).

For instance, you could create a prompt that writes a summary of an article and then pass in the url of any article on the internet. We'll fetch the article and pass it to the AI platform as context along with the prompt.

This is a very powerful feature that we're constantly working on improving. To get started with web scraping, you'll need a Pro plan or high tier.