Web Scraping: A Big Data Building Tool And Its Status In The Fintech Sector In Viet Nam

Abstract

The amount of data in our lives is growing exponentially. With this surge, data analytics has become a hugely important part of the way organizations are run. And while data has many sources, its biggest repository is on the web. As the fields of big data analytics, artificial intelligence and machine learning grow, companies need data analysts who can scrape the web in increasingly sophisticated ways.

Web scraping (or data scraping) is a technique used to collect content and data from the internet. This data is usually saved in a local file so that it can be manipulated and analyzed as needed. If you've ever copied and pasted content from website into a an Excel spreadsheet, this is essentially what web scraping is, but on a very small scale. This paper will focus on various aspects of web scraping, beginning with the basic introduction and a brief discussion on various software's and tools for web scrapping. There are many benefits of web scraping in fintech sector such as market sentiment, compliance, risk mitigation, monitoring ratings changes, understanding customers, etc. The author had also found out some outstanding web scraping tools with some strengths and weaknesses and finally concluded with the web scraping's current status of application in the fintech sector in Vietnam.