Відео

Wow mind blown. Amazing

Great video, thanks. New subscriber (and like) here. I had a couple of questions though: why use langchain? It seems unnecessary from what I have read. Would also love a demo ipynb/copy of code.

10:42 i follow tutorial, build scraper with cleanmymac, nothing happen, install twice, Ubuntu 22.04 only get many index.html

Extremely interesting!

Jason, I watch a lot of AI videos but I learn the most from yours. I am actually excited everytime i see you have put another one out. Keep up the great work!

With great respect to you that you extend the explanation in an exaggerated way.

I have the same idea in mind. I have tons of product documents that I wish I could just ask an agent something about it instead of scrolling hundreds of word pages. I really appreciate your video man.

wow nice work thanks!

are there any hidden charges?

Any TLDR here for this nightmare long blob video?

I am already doing this. Its the same way I trained models to play video games - take a screensshot, convert to greyscale, but instead of inserting that into a CNN, I pipe it into an agent that I built and it has mouse and keyboard tools instead of the typical selenium/headless tools. It works pretty damn good although some models will refuse cpatchas outright.

Who wants to be a millionare? Scrape linkedin with AI and become a Zoominfo competitor. Youre welcome.

Попробуй любую рифму

手部修复模型有点奇怪，后面可以修复吗。

I was able to follow up until generating the face image, then all the IPadapter nodes and preview bridge nodes are not showing up. I'm not sure if I'm doing something wrong or its just that the nodes are broken (as some say in the comments) would love to know whats the problem here. thanks!!

You didn't name the title of the speech, the names of the authors or team, got to give credit where it's due... can we get a link to the videos your using? the source? i would like to see the whole thing

scraping a commercial website is called "prison oriented programming" in China. It's illegal.

is there an offline ai model that can help me in research study and work? i am looking for one thank you i am not as adept on this :(

Hi, Im building a PDF QA chatbot than answers from 10 long pdfs, Ive experimented with RAG but the chunks I get from the vector db often dont provide the correct context, what can I do to get reliable answers based on my pdfs? will passing the entirety of the pdfs to an llm with a large max tokens help? it doesnt seem effecient to pass the entirety of the pdfs with every question ask.... Im lost please help

Hi Jason, thanks for your interesting video. Would it be possible to place your microphon so that we can see your lips when you are talking. For me it's easier to understand english, if I can see them. You huge mic covers so much of your face. Thanks.

I wonder if this is an Advertisement video or a knowledge sharing video..Nothing is open source.

We are in a world where data is the most sought after commodity. And AI is going to make accessing information trivial. I wonder how Big Business will respond. I suspect they'll start pushing for laws to criminalize web scraping in the not too distant future. It will be interesting to see how this all plays out in the years to come.

My whole startup is based on scraping. I hope this doesn't catch up...

why are you building a startup every week

🏆

Is bounding box method open-source ? Looking for a function that returns an X,Y coordinate of an element. Def FindCoordinates(instruction, screenshot) Return (x coordinate, y coordonate)

This looks sus selenium could do this why do all this work if gpt 4o is generating selenium scripts faster than an Asian thinking

can it be used for scrapping estate agents for finding a house to buy?

do you recommend reading the robots.txt first?

Great work!! I'm currently tackling web scraping challenges, especially with certain sites where determining the delivery location or dealing with pop-ups obstructing the content poses issues. This often requires user action before the search query can proceed. What do you believe are the most effective methods or tools to overcome these hurdles? Sometimes, even the agentql struggle to resolve these issues.

Good video. I wonder, how did you get prompts from flowgpt? seems not to show the actual prompts anymore

With all these expensive tools, I think it will best to build with playwright. Though it will take weeks or months, but it will be cost effective.

I'm here for the thumbnail

This is extremely helpful! Awesome!

🎯 Key Takeaways for quick navigation: 00:00 *🌐 Amount of Data Created on the Internet* - Huge amount of new data is created on the internet annually. - 147 zitabytes of data estimated by the end of 2024. - Websites generate massive data, with 252,000 new websites created daily. 01:10 *🤖 Web Scraping Overview* - Bots and computers scrape valuable information from websites. - Scraping involves mimicking web browser behavior to retrieve data. - Websites often lack APIs, making scraping the main method to extract structured data. 03:14 *🖥️ Challenges in Web Scraping* - Websites are designed for user experience, not machine access. - Techniques like progressive loading and authentication hinder scraping. - Solutions include using headless browsers and simulating human behavior. 06:17 *🧠 Benefits of Large Language Models in Web Scraping* - Large language models excel at handling unstructured data. - Multi-model models like GPT-4V bridge the gap between human and machine browsing. - Models extract structured data uniformly from diverse website structures. 08:21 *🌀 Advanced Web Scraping Agents* - Agents with multimodal capabilities can control web browsers and extract data. - Platforms like Multi-on and HyperRDE create sophisticated web browsing agents. - Universal web scraping agents can process natural language prompts to extract data from any source. 21:24 *🤖 Building a conversational agent with memory optimization* - Creating a function to log agent's actions and results - Implementing memory optimization to prevent exceeding the token context window limit - Defining a process for the agent to make a plan before taking action 22:35 *🕵️‍♂️ Dividing the research workflow for the agent* - Setting up a two-stage workflow for the agent: website search and internet search - Specifying the data points to collect from websites like Discord - Providing an example of running the agent to collect data about companies offering employee catering services 25:22 *🛒 Developing a universal e-commerce website scrapper* - Using Agent ql library to set up a universal e-commerce website scrapper - Defining queries and actions for the scrapper, such as extracting product information and navigating pagination - Demonstrating the scrapper's functionality by collecting product data from multiple pages on e-commerce websites Made with HARPA AI

Came to train my 3 Llamas... Now I'm a full stack developer.

Fire video🔥🔥🔥

Hi I am a student, I want to build same kind of thing for LinkedIn can it possible. Thank you so much for sharing this knowledge❤

perplexity should use this crawler since their models are hallucinating reference URLs LOL

Your content is fantastic, your English is top-notch but your accent adds some overhead to understanding. I hope that doesn't feel insulting, your vocabulary and grammar is better than most native English speakers. So an idea... Could you look at using gpt-4o to improve elecution (not just English) in a foreign language? It would be quite useful for many people.

Hi thank you, great video. I'm new to coding and I get lost at the setup. any tutorials on how to set this up step by step ?

Total nonsense

In movies they do all they can so the AI cannot access the internet, in real life : we need web scrapping man, give it access!

Would it be possible for me to contribute and collaborate on this project? I’m also working on developing a universal scraper myself.

You talked about 'universal scrapers' then you used a bunch of expensive services to create a very vanilla hyper-specific scraper that doesn't' require LLMs at all.... hmm....

hey can u make a video on Multion

list your app in there is an AI for that and product hunt and hacker news!!! I am sure you got a great thing in your hands!!!

Excelente. Gracias.

If you are interested in universal web scraper i'm building, please leave your email in this waiting list: forms.gle/8xaWBBfR9EL5w8jr6

You didn't put the signup link for your universal scraper agent.