How to Scrape Google Flights: Extract Prices, Airlines, and Schedules with Python

TL;DR

• Google Flights is a valuable source for flight information, but is difficult to scrape due to its JavaScript-heavy pages and anti-bot protections
• Use Playwright to render JavaScript-heavy pages
• Use Pydantic to structure and validate data
• Use rotating residential proxies to avoid blocks
• Organize your project into multiple files for maintainability
• Export scraped results to JSON for analysis

Prerequisites and environment setup

Before you scrape Google Flights, you need to prepare a Python environment and install the required libraries. This step makes your web scraper run in a controlled environment and prevents dependency conflicts with other Python web scraping projects.

Because Google aggressively monitors automated traffic, scraping flight information without the proper infrastructure will likely result in blocked requests. To avoid this, we'll configure residential proxies that rotate IP addresses and simulate real users accessing the platform.

Preparing a Python environment

You must have the required Python version before you can start the web scraping process.

This is because most modern scraping libraries rely on recent Python versions for performance improvements and compatibility with asynchronous frameworks. Using outdated Python versions can cause dependency errors or missing features.

You'll need the latest version of Python installed on your computer.

For this project, we'll use Python 3.9+, which supports asynchronous programming and modern packaging tools.

But first:

Check your Python version:

On Windows: open the Command Prompt and type:

python --version or py –version

python --version or py –version

On macOS and Linux: open the Terminal and type:

python --version or python3 --version 

python --version or python3 --version 

If you don't have the latest version of Python on your system, install it by visiting the downloads page from the official website before continuing.

Create a virtual environment:

Next, create a virtual environment and activate it to isolate dependencies for individual projects.

This prevents library conflicts between projects and ensures reproducibility when sharing code with other developers. It's also considered a best practice in Python development.

Use:

python -m venv flights_scraper_env

python -m venv flights_scraper_env

Then activate the virtual environment by running these commands:

On Mac/Linux:

source flights_scraper_env/bin/activate

source flights_scraper_env/bin/activate

On Windows:

flights_scraper_env\Scripts\activate

flights_scraper_env\Scripts\activate

Install required dependencies:

To scrape Google Flights effectively, you need tools that can render dynamic web pages and structure extracted data. 

We'll use several Python libraries, including:

• Playwright – browser automation for dynamic pages
• Pydantic – define and validate structured data models
• Python-dotenv – securely manage environment variables like proxy credentials

These libraries work together to create a reliable scraping pipeline. Playwright loads JavaScript-rendered pages, Pydantic validates the scraped data, and dotenv ensures sensitive information remains outside the source code.

Install the three libraries by running the following command:

pip install playwright pydantic python-dotenv

pip install playwright pydantic python-dotenv

Next, install the Chromium browser binaries required by Playwright:

npx playwright install chromium

npx playwright install chromium

This command will download and install a headless Chromium browser that Playwright needs for running automated tests on Google Flights.

Setting up proxies

Google's anti-bot systems monitor traffic patterns, IP addresses, and request frequency. If multiple requests originate from the same IP address in a short period, Google may block the connection or display CAPTCHA challenges. 

You'll need to use proxies to navigate around all this.

The two common types of proxies you can use for this project are rotating datacenter proxies and rotating residential proxies.

While datacenter proxies are high-speed and cost-effective, they send requests through server/cloud IP addresses owned by hosting providers (not real consumer ISPs). Google systems can also identify these proxies as automated infrastructure and block them or display CAPTCHA challenges when you use them for large-scale scraping.

Rotating residential proxies can solve this. These proxies support geo-targeting. You can route requests through IPs in specific countries, which is critical for capturing region-specific flight pricing (more on this in a later section). This reduces IP bans, CAPTCHAs or request blocks.

Setting up Decodo residential proxies

Residential proxies allow you to simulate requests from real users around the world. This is particularly useful when scraping flight prices as they vary depending on the user's geographic location.

Decodo provides residential proxies for web scraping and large-scale data collection. These proxies use endpoints. When you connect to an endpoint, your traffic is routed through a random IP address. You get high anonymity so you can avoid rate limits and enable geo-targeted searches. They also offer accurate geo-location targeting across over 195 locations.

Here is a request example of Decodo residential proxies in Python:

import requests
url = 'https://ip.decodo.com/json'
username = 'username'
password = 'password'
proxy = f"http://{username}:{password}@gate.decodo.com:7000"
result = requests.get(url, proxies = {
    'http': proxy,
    'https': proxy
})
print(result.text)

import requests
url = 'https://ip.decodo.com/json'
username = 'username'
password = 'password'
proxy = f"http://{username}:{password}@gate.decodo.com:7000"
result = requests.get(url, proxies = {
    'http': proxy,
    'https': proxy
})
print(result.text)

Follow these simple steps to set up rotating residential proxies with Decodo:

• Go to the Residential → Proxy setup page and log in to the dashboard
• Select "Residential" proxies and choose a plan
• Go to the parameter selection section below your authentication methods

• Set the location of your proxy

• Choose your preferred Session type (Sticky or Rotating)

• Select your preferred Protocol format: HTTP(S) or SOCKS5

From there, use the generated username and password for authentication to integrate with browsers or scrapers.

You can also watch this video guide to learn how to set up and use Decodo residential proxies.

Storing proxy credentials securely

Hardcoding credentials inside source code can cause a security risk and make your project harder to maintain. Instead, store sensitive information securely in environment variables to keep it separate from the application code.

Create a .env file in your project root directory (alongside your Python files) to store your credentials locally and keep them excluded from version control systems like Git.

PROXY_HOST=gate.decodo.com
PROXY_PORT=10000
PROXY_USER=username
PROXY_PASSWORD=password

PROXY_HOST=gate.decodo.com
PROXY_PORT=10000
PROXY_USER=username
PROXY_PASSWORD=password

Then, at the top of proxy_manager.py (which you will create in the next section), load the variables like this:

# proxy_manager.py


from dotenv import load_dotenv
import os

load_dotenv()

proxy_host = os.getenv("PROXY_HOST")
proxy_port = os.getenv("PROXY_PORT")
proxy_user = os.getenv("PROXY_USER")
proxy_password = os.getenv("PROXY_PASSWORD")

# proxy_manager.py


from dotenv import load_dotenv
import os

load_dotenv()

proxy_host = os.getenv("PROXY_HOST")
proxy_port = os.getenv("PROXY_PORT")
proxy_user = os.getenv("PROXY_USER")
proxy_password = os.getenv("PROXY_PASSWORD")

Choose a project structure

Organize your project into multiple files to improve readability and simplify maintenance.

Use a clean project structure to separate models, scraping logic, proxy handling, and execution scripts. 

This modular design is especially useful when scaling the scraper or adding additional features later.

flights-scraper/
├── .env               # Proxy credentials (never commit this)
├── models.py          # Pydantic data models
├── proxy_manager.py   # Proxy configuration logic
├── scraper.py         # Core scraping logic
└── main.py            # Entry point and orchestration

flights-scraper/
├── .env               # Proxy credentials (never commit this)
├── models.py          # Pydantic data models
├── proxy_manager.py   # Proxy configuration logic
├── scraper.py         # Core scraping logic
└── main.py            # Entry point and orchestration

Here is the purpose of each file:

• models.py: Defines data models for search parameters and results.
• scraper.py: Contains the Playwright scraping logic.
• proxy_manager.py: Handles proxy rotation and configuration.
• main.py: Runs the scraper and saves results.

Defining data models for flight information

When scraping flight data, raw dictionaries can quickly become messy.

Data models define the structure of both the input parameters and the scraped results, enforce consistent data formats, and simplify validation. This approach reduces errors when processing large datasets.

We'll use Pydantic, a popular Python library designed for data validation and JSON serialization. With this data model, the scraped data matches the expected schema and automatically converts values into appropriate types.

Configuration model

The configuration model stores the parameters you use to generate flight searches. These parameters include origin city/airport, destination city/airport, departure date, return date (optional), trip type (one-way vs. round-trip), target currencies, list of proxy countries to scrape from.

The model also makes the scraper easier to reuse. Instead of editing code every time you change routes or dates, you simply update the configuration object.

Here is an example configuration model using Pydantic:

from pydantic import BaseModel
from datetime import date
from enum import Enum
from typing import Optional, List

from pydantic import BaseModel
from datetime import date
from enum import Enum
from typing import Optional, List

Using Python Enums for fixed values like trip type:

class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"

class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"

Add the following code to models.py:

# models.py


from datetime import date
from typing import Optional, List
from enum import Enum
from pydantic import BaseModel, field_validator, model_validator
class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"
class FlightSearchConfig(BaseModel):
    origin: str
    destination: str
    departure_date: date
    return_date: Optional[date] = None
    trip_type: TripType
    currency: str = "USD"
    proxy_countries: List[str]
    @field_validator("departure_date")
    @classmethod
    def departure_must_be_future(cls, value: date):
        if value <= date.today():
            raise ValueError("Departure date must be in the future")
        return value
    @field_validator("proxy_countries")
    @classmethod
    def proxy_list_not_empty(cls, value):
        if not value:
            raise ValueError("proxy_countries cannot be empty")
        return value
    @model_validator(mode="after")
    def validate_trip_logic(self):
        if self.trip_type == TripType.ROUND_TRIP and not self.return_date:
            raise ValueError("Round trip requires a return date")
        if self.trip_type == TripType.ONE_WAY and self.return_date:
            raise ValueError("One-way trip cannot have a return date")
        if self.return_date and self.return_date <= self.departure_date:
            raise ValueError("Return date must be after departure date")
        return self

# models.py


from datetime import date
from typing import Optional, List
from enum import Enum
from pydantic import BaseModel, field_validator, model_validator
class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"
class FlightSearchConfig(BaseModel):
    origin: str
    destination: str
    departure_date: date
    return_date: Optional[date] = None
    trip_type: TripType
    currency: str = "USD"
    proxy_countries: List[str]
    @field_validator("departure_date")
    @classmethod
    def departure_must_be_future(cls, value: date):
        if value <= date.today():
            raise ValueError("Departure date must be in the future")
        return value
    @field_validator("proxy_countries")
    @classmethod
    def proxy_list_not_empty(cls, value):
        if not value:
            raise ValueError("proxy_countries cannot be empty")
        return value
    @model_validator(mode="after")
    def validate_trip_logic(self):
        if self.trip_type == TripType.ROUND_TRIP and not self.return_date:
            raise ValueError("Round trip requires a return date")
        if self.trip_type == TripType.ONE_WAY and self.return_date:
            raise ValueError("One-way trip cannot have a return date")
        if self.return_date and self.return_date <= self.departure_date:
            raise ValueError("Return date must be after departure date")
        return self

We'll explain the fields for the above code briefly so you can understand this better:

Date validation makes sure:

• Departure dates are valid
• Return date follows departure date
• Currency format remains consistent

Flight result model

Now, let's define the structure for scraped data.

The flight result model defines how extracted data will be stored. Each scraped flight becomes a structured object containing fields such as:

• Airline name
• Departure time
• Arrival time
• Flight duration
• Number of stops
• Layover details, price (as a string with currency symbol)
• Emissions data
• Proxy country used and scrape timestamp.

As shown below:

class FlightResult(BaseModel):
    airline: str
    departure_time: str
    arrival_time: str
    duration: str
    stops: int
    layover_details: Optional[str] = None
    price: str                         # e.g., "$312"
    emissions: Optional[str] = None   # e.g., "176 kg CO2e"
    proxy_country: str
    scraped_at: datetime = datetime.utcnow()

class FlightResult(BaseModel):
    airline: str
    departure_time: str
    arrival_time: str
    duration: str
    stops: int
    layover_details: Optional[str] = None
    price: str                         # e.g., "$312"
    emissions: Optional[str] = None   # e.g., "176 kg CO2e"
    proxy_country: str
    scraped_at: datetime = datetime.utcnow()

You will also need to add a method to serialize scraped flight results to JSON-friendly dictionaries. This is important for interoperability, data storage, and efficient data exchange across different systems and programming languages.

    def to_dict(self) -> dict:
        return self.model_dump(mode="json")

    def to_dict(self) -> dict:
        return self.model_dump(mode="json")

Here is the complete code for models.py, which defines all the data models for flight information:

# models.py
from pydantic import BaseModel, Field, field_validator, model_validator
from typing import Optional, List
from datetime import date, datetime, timezone
from enum import Enum
class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"
class SearchConfig(BaseModel):
    origin: str
    destination: str
    departure_date: date
    return_date: Optional[date] = None
    trip_type: TripType = TripType.ONE_WAY
    currency: str = "USD"
    proxy_countries: List[str] = Field(default_factory=lambda: ["US"])
    @field_validator("origin", "destination")
    @classmethod
    def validate_airport(cls, v: str) -> str:
        v = v.upper()
        if len(v) != 3 or not v.isalpha():
            raise ValueError("Airport code must be a 3-letter IATA code.")
        return v
    @field_validator("departure_date")
    @classmethod
    def departure_must_be_future(cls, v: date) -> date:
        if v < date.today():
            raise ValueError("Departure date must be in the future.")
        return v
    @field_validator("currency")
    @classmethod
    def currency_uppercase(cls, v: str) -> str:
        v = v.upper()
        if len(v) != 3 or not v.isalpha():
            raise ValueError("Currency must be a valid 3-letter ISO code.")
        return v
    @model_validator(mode="after")
    def validate_trip(self):
        if self.return_date and self.return_date <= self.departure_date:
            raise ValueError("Return date must be after departure date.")
        if self.trip_type == TripType.ROUND_TRIP and not self.return_date:
            raise ValueError("Return date required for round trip.")
        return self
class FlightResult(BaseModel):
    airline: str
    departure_time: datetime
    arrival_time: datetime
    duration_minutes: int
    stops: int
    layover_details: Optional[str] = None
    price: float
    emissions: Optional[str] = None
    proxy_country: str
    scraped_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
def to_dict(self) -> dict:
        return self.model_dump(mode="json")

# models.py
from pydantic import BaseModel, Field, field_validator, model_validator
from typing import Optional, List
from datetime import date, datetime, timezone
from enum import Enum
class TripType(str, Enum):
    ONE_WAY = "one_way"
    ROUND_TRIP = "round_trip"
class SearchConfig(BaseModel):
    origin: str
    destination: str
    departure_date: date
    return_date: Optional[date] = None
    trip_type: TripType = TripType.ONE_WAY
    currency: str = "USD"
    proxy_countries: List[str] = Field(default_factory=lambda: ["US"])
    @field_validator("origin", "destination")
    @classmethod
    def validate_airport(cls, v: str) -> str:
        v = v.upper()
        if len(v) != 3 or not v.isalpha():
            raise ValueError("Airport code must be a 3-letter IATA code.")
        return v
    @field_validator("departure_date")
    @classmethod
    def departure_must_be_future(cls, v: date) -> date:
        if v < date.today():
            raise ValueError("Departure date must be in the future.")
        return v
    @field_validator("currency")
    @classmethod
    def currency_uppercase(cls, v: str) -> str:
        v = v.upper()
        if len(v) != 3 or not v.isalpha():
            raise ValueError("Currency must be a valid 3-letter ISO code.")
        return v
    @model_validator(mode="after")
    def validate_trip(self):
        if self.return_date and self.return_date <= self.departure_date:
            raise ValueError("Return date must be after departure date.")
        if self.trip_type == TripType.ROUND_TRIP and not self.return_date:
            raise ValueError("Return date required for round trip.")
        return self
class FlightResult(BaseModel):
    airline: str
    departure_time: datetime
    arrival_time: datetime
    duration_minutes: int
    stops: int
    layover_details: Optional[str] = None
    price: float
    emissions: Optional[str] = None
    proxy_country: str
    scraped_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
def to_dict(self) -> dict:
        return self.model_dump(mode="json")

Why Pydantic is better than raw dictionaries

Using raw dictionaries for scraped data may seem convenient initially. However, dictionaries don't enforce data types or validate values, which can lead to inconsistent datasets. Over time, this can cause significant issues when analyzing or exporting the data.

Here is why Pydantic is the better choice:

• Automatic type validation catches malformed data early
• Clear schema makes the codebase self-documenting
• Easy serialization to JSON with .model_dump()

Building dynamic search URLs for Google Flights

Before you build the scraper, you need a reliable way to construct Google Flights search URLs. Google Flights encodes search parameters directly in the URL. If you get the URL right, the page loads with results already populated without requiring any form submissions.

Note: You will need to add the following codes inside scraper.py:

Anatomy of a Google Flights URL

A typical Google Flights URL looks like this:

https://www.google.com/travel/flights?tfs=...

https://www.google.com/travel/flights?tfs=...

There are two ways to search Google Flights through the URL:

• Natural-language query approach: Uses ?q= …, for example, ?q=flights from X to Y. A simple flight search may look like this: https://www.google.com/travel/flights?q=Flights%20from%20NBO%20to%20DXB

This URL structure is simpler and works in a browser, but is less reliable for automated scraping because results can be inconsistent. It also requires minimal encoding but offers less control over advanced parameters.

You can use this structure for small scraping projects.

• Structured parameter approach: Uses ?tfs= … for example, https://www.google.com/travel/flights?tfs=... This is the encoded format Google uses internally. It encodes origin, destination, dates, and trip type into a compact string. This is more stable for automated scraping and offers a more precise search configuration. However, it's also hard to generate programmatically and requires reverse engineering Google's encoding format.

We'll use the natural-language query approach for simplicity and reliability at a moderate scale.

URL encoding considerations

City names or routes may contain spaces or special characters. You need to URL-encode these before you use them in the query string.

For example, using the natural-language query approach: 

Flights from New York to London

Flights from New York to London

Becomes:

https://www.google.com/travel/flights?q=Flights%20from%20New%20York%20to%20London

https://www.google.com/travel/flights?q=Flights%20from%20New%20York%20to%20London

Python provides built-in urllib utilities to handle this automatically.

For example:

from urllib.parse import quote
query = "Flights from NBO to DXB on 2025-06-10"
encoded_query = quote(query)
url = f"https://www.google.com/travel/flights?q={encoded_query}"
print(url)

from urllib.parse import quote
query = "Flights from NBO to DXB on 2025-06-10"
encoded_query = quote(query)
url = f"https://www.google.com/travel/flights?q={encoded_query}"
print(url)

The Python code above will generate a Google Flights search URL for the query “Flights from NBO to DXB on 2025-06-10”.

Building one-way vs. round-trip Google Flights URLs

The query format slightly changes depending on the trip type. One-way example:

def build_one_way_url(origin, destination, departure_date):
    query = f"Flights from {origin} to {destination} on {departure_date}"
    return f"https://www.google.com/travel/flights?q={quote(query)}"

def build_one_way_url(origin, destination, departure_date):
    query = f"Flights from {origin} to {destination} on {departure_date}"
    return f"https://www.google.com/travel/flights?q={quote(query)}"

This function builds a Google Flights search URL for a one-way trip.

Round-trip example:

from urllib.parse import quote
def build_round_trip_url(origin, destination, departure_date, return_date):
    query = (
        f"Round trip flights from {origin} to {destination} "
        f"departing {departure_date} returning {return_date}"
    )
    return f"https://www.google.com/travel/flights?q={quote(query)}"

from urllib.parse import quote
def build_round_trip_url(origin, destination, departure_date, return_date):
    query = (
        f"Round trip flights from {origin} to {destination} "
        f"departing {departure_date} returning {return_date}"
    )
    return f"https://www.google.com/travel/flights?q={quote(query)}"

This function builds a Google Flights search URL for a round-trip.

Setting currency with the curr parameter

Google Flights will show prices in the default currency for the user's region, and this can make it harder to compare results scraped from different geographic locations.

You need to specify the currency using the curr parameter for the scraper to extract flight prices in a consistent currency.

This will get the scraper to extract prices in a consistent currency.

For example, this URL will show flights that use the USD currency:

https://www.google.com/travel/flights?q=Flights%20from%20NBO%20to%20DXB&curr=USD

https://www.google.com/travel/flights?q=Flights%20from%20NBO%20to%20DXB&curr=USD

Here are practical tips before you run automated scraping projects with your dynamic search URLs for Google Flights:

• Always test generated URLs manually in a browser before relying on them in an automated scrape. Paste the URL into Chrome and verify that the results page loads correctly.
• Google Flights occasionally changes its URL structure. Log the URLs your scraper generates alongside the results. If results suddenly drop to zero, you can inspect recent URLs to catch breaking changes early.
• Use URL-encoding for city names with spaces or special characters. The urlencode() function handles this automatically, but double-check names by testing them manually.
• For round-trip searches, include both departure and return dates in the query string.

Building the proxy manager

The proxy manager handles how Decodo residential proxy credentials are formatted and injected into each browser session. It uses the geo-targeted residential proxy endpoints to route each request through a specific country.

Create proxy_manager.py:

# proxy_manager.py


import os
from dotenv import load_dotenv
load_dotenv()
class ProxyManager:
    """
    Manages Decodo residential proxy configuration.
    Formats proxy credentials for use with Playwright.
    """
    def __init__(self):
        self.host = os.getenv("PROXY_HOST", "gate.decodo.com")
        self.port = int(os.getenv("PROXY_PORT", "10000"))
        self.user = os.getenv("PROXY_USER")
        self.password = os.getenv("PROXY_PASSWORD")
        if not self.user or not self.password:
            raise ValueError("Proxy credentials not found. Check your .env file.")
    def get_proxy(self, country_code: str) -> dict:
        """
        Returns a Playwright-compatible proxy config for a given country.
        Decodo geo-targeting uses country-specific subdomains.
        """
        # Decodo country-specific endpoint format
        host = f"{country_code.lower()}.{self.host}"
        return {
            "server": f"http://{host}:{self.port}",
            "username": self.user,
            "password": self.password
        }

# proxy_manager.py


import os
from dotenv import load_dotenv
load_dotenv()
class ProxyManager:
    """
    Manages Decodo residential proxy configuration.
    Formats proxy credentials for use with Playwright.
    """
    def __init__(self):
        self.host = os.getenv("PROXY_HOST", "gate.decodo.com")
        self.port = int(os.getenv("PROXY_PORT", "10000"))
        self.user = os.getenv("PROXY_USER")
        self.password = os.getenv("PROXY_PASSWORD")
        if not self.user or not self.password:
            raise ValueError("Proxy credentials not found. Check your .env file.")
    def get_proxy(self, country_code: str) -> dict:
        """
        Returns a Playwright-compatible proxy config for a given country.
        Decodo geo-targeting uses country-specific subdomains.
        """
        # Decodo country-specific endpoint format
        host = f"{country_code.lower()}.{self.host}"
        return {
            "server": f"http://{host}:{self.port}",
            "username": self.user,
            "password": self.password
        }

This proxy manager formats the Decodo endpoint for geo-targeted routing so that each country gets its own browser session. This is to enable the scraper to fetch flight prices from the correct regional perspective.

Initializing the scraper and launching Playwright

Your Google Flights search URLs are ready to go, so let's build the core scraper.

We'll use Playwright because it can scrape websites with dynamic content and simulate real user browsing behavior. We'll also configure it with proxy settings from your proxy manager.

Scraper class structure

Create scraper.py and define a main GoogleFlightsScraper class that organizes the different responsibilities: URL generation, browser management, data extraction, and results aggregation.

Keeping these as separate methods makes the code easier to debug, test, and extend.

Here is the scraper class structure in Python that accepts a proxy manager and search config

# scraper.py


class GoogleFlightsScraper:
    def __init__(self, config, proxy_manager):
        self.config = config
        self.proxy_manager = proxy_manager
    async def build_search_url(self):
        pass
    async def launch_browser(self):
        pass
    async def extract_flights(self, page):
        pass
    async def run(self):
        pass

# scraper.py


class GoogleFlightsScraper:
    def __init__(self, config, proxy_manager):
        self.config = config
        self.proxy_manager = proxy_manager
    async def build_search_url(self):
        pass
    async def launch_browser(self):
        pass
    async def extract_flights(self, page):
        pass
    async def run(self):
        pass

Launching Playwright

Playwright supports asynchronous execution, which improves performance when scraping JavaScript-heavy pages like Google Flights.

Start by initializing the Playwright runtime. We'll use async_playwright() for asynchronous execution, which allows multiple scraping tasks to run concurrently if needed.

Add the following logic inside scraper.py inside the launch_browser method.

from playwright.async_api import async_playwright


async with async_playwright() as p:
    browser = await p.chromium.launch(headless=True)

from playwright.async_api import async_playwright


async with async_playwright() as p:
    browser = await p.chromium.launch(headless=True)

Next, configure the headless Chromium browser with proxy settings passed with Decodo credentials to reduce automation detection.

browser = await p.chromium.launch(
    headless=True,
    proxy={
        "server": f"http://{proxy_host}:{proxy_port}",
        "username": proxy_user,
        "password": proxy_password
    },
    args=[
        "--disable-blink-features=AutomationControlled",
        "--no-sandbox"
    ]
)

browser = await p.chromium.launch(
    headless=True,
    proxy={
        "server": f"http://{proxy_host}:{proxy_port}",
        "username": proxy_user,
        "password": proxy_password
    },
    args=[
        "--disable-blink-features=AutomationControlled",
        "--no-sandbox"
    ]
)

The complete function will look like this:

# scraper.py


from playwright.async_api import async_playwright
async with async_playwright() as p:
    browser = await p.chromium.launch(
        headless=True,
        proxy={
            "server": proxy["server"],
            "username": proxy["username"],
            "password": proxy["password"]
        },
        args=[
            "--disable-blink-features=AutomationControlled",
            "--no-sandbox"
        ]
    )

# scraper.py


from playwright.async_api import async_playwright
async with async_playwright() as p:
    browser = await p.chromium.launch(
        headless=True,
        proxy={
            "server": proxy["server"],
            "username": proxy["username"],
            "password": proxy["password"]
        },
        args=[
            "--disable-blink-features=AutomationControlled",
            "--no-sandbox"
        ]
    )

This way, requests go through residential IPs, and authentication is handled at the browser level, so Google treats sessions as real users.

Applying stealth and anti-detection measures

Even with proxies in place, headless browsers have distinctive characteristics that Google's anti-bot systems can detect. Configure your browser environment to make your sessions look more like real users.

Set a realistic user agent and viewport dimensions to match common screen sizes:

context = await browser.new_context(
    user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
    viewport={"width": 1366, "height": 768}
)

context = await browser.new_context(
    user_agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
    viewport={"width": 1366, "height": 768}
)

You will need to block unnecessary resources such as images, fonts, and stylesheets to make pages load quickly and speed up scraping.

await context.route(
    "**/*",
    lambda route: route.abort()
    if route.request.resource_type in ["image", "font", "stylesheet"]
    else route.continue_()
)

await context.route(
    "**/*",
    lambda route: route.abort()
    if route.request.resource_type in ["image", "font", "stylesheet"]
    else route.continue_()
)

Automated scripts that run too quickly can trigger anti-bot systems. You need to add randomized delays between actions to mimic human browsing patterns. 

Example of a randomized delay function:

import random
import asyncio
async def random_sleep():
    delay = random.uniform(3, 6)
    print(f"Sleeping for {delay:.2f} seconds...")
    await asyncio.sleep(delay)
    print("Done sleeping!")
# Run the async function
asyncio.run(random_sleep())

import random
import asyncio
async def random_sleep():
    delay = random.uniform(3, 6)
    print(f"Sleeping for {delay:.2f} seconds...")
    await asyncio.sleep(delay)
    print("Done sleeping!")
# Run the async function
asyncio.run(random_sleep())

This function simulates human browsing patterns.

Page navigation

You have configured the browser. Now, open a new page and navigate to the generated search URL.

page = await context.new_page()
await page.goto(
    search_url,
    wait_until="domcontentloaded",
    timeout=60000
)

page = await context.new_page()
await page.goto(
    search_url,
    wait_until="domcontentloaded",
    timeout=60000
)

Then wait for the flight results to appear.

await page.wait_for_selector("div[jsname]")

await page.wait_for_selector("div[jsname]")

This will get the page to finish rendering before data extraction begins.

If the selector never appears, handle the error gracefully using the following function:

try:
    await page.wait_for_selector("div[jsname]", timeout=15000)
except:
    print("No flight results found or page failed to load.")

try:
    await page.wait_for_selector("div[jsname]", timeout=15000)
except:
    print("No flight results found or page failed to load.")

Extracting flight data from the page

With the browser running and the Google Flights results page loaded, the next step is to parse the page and pull out the structured flight data you need.

Google Flights dynamically generates its HTML, so you need to choose the right selector.

Identify flight card elements

Google Flights renders its results as a list of flight cards. Each card contains the core details for a single flight option. The specific CSS selectors Google uses can change over time, but the general DOM structure has remained relatively consistent.

Start by inspecting the page using browser developer tools.

Note that Google uses obfuscated, short, and randomized CSS class names (e.g., pIav2d, YMlIz, sSHqwe) generated at build time to significantly reduce file sizes, prevent CSS naming conflicts in large applications, and hinder web scraping. 

You can select all result cards with:

flight_cards = await page.query_selector_all("div.pIav2d")

flight_cards = await page.query_selector_all("div.pIav2d")

These classes often change frequently because they are hashed based on component structure or updated during continuous deployment, breaking static scraping bots.

You need to build selectors that rely on structural position and ARIA attributes where possible, and verify selectors regularly.

Extracting individual data points

Each flight card contains multiple nested elements. Let's see how to extract each data point with Playwright.

Add this helper function inside scraper.py:

Price

price_element = await card.query_selector(".YMlIz")
if price_element:
    price = await price_element.inner_text()
else:
    price = None

price_element = await card.query_selector(".YMlIz")
if price_element:
    price = await price_element.inner_text()
else:
    price = None

Departure and arrival times

time_elements = await card.query_selector_all("span")
if len(time_elements) >= 2:
    departure_time = (await time_elements[0].inner_text()).strip()
    arrival_time = (await time_elements[1].inner_text()).strip()
else:
    departure_time = arrival_time = None  # Or handle error

time_elements = await card.query_selector_all("span")
if len(time_elements) >= 2:
    departure_time = (await time_elements[0].inner_text()).strip()
    arrival_time = (await time_elements[1].inner_text()).strip()
else:
    departure_time = arrival_time = None  # Or handle error

Flight duration

duration = await card.query_selector(".gvkrdb").inner_text()

duration = await card.query_selector(".gvkrdb").inner_text()

Number of stops

The stop information typically appears as text:

Nonstop

1 stop

2 stops

To extract the number of stops:

stops_text = await card.query_selector(".EfT7Ae").inner_text()
if "Nonstop" in stops_text:
    stops = 0
else:
    stops = int(stops_text.split()[0])

stops_text = await card.query_selector(".EfT7Ae").inner_text()
if "Nonstop" in stops_text:
    stops = 0
else:
    stops = int(stops_text.split()[0])

Airline name

airline = await card.query_selector(".sSHqwe").inner_text()

airline = await card.query_selector(".sSHqwe").inner_text()

Emissions data

Some flights display estimated emissions.

Extract it using:

emissions_element = await card.query_selector(".V1iAHe")
emissions = None
if emissions_element:
    emissions = await emissions_element.inner_text()

emissions_element = await card.query_selector(".V1iAHe")
emissions = None
if emissions_element:
    emissions = await emissions_element.inner_text()

Error handling

Google Flights pages can vary depending on the route. Wrap each extraction step in try/except blocks to handle missing elements gracefully and avoid crashing the scraper.

Use:

try:
    price = await price_element.inner_text()
except:
    price = None

try:
    price = await price_element.inner_text()
except:
    price = None

Returning None for missing fields will keep the dataset usable.

For efficiency, limit extraction to the first 10–15 results, which usually contain the most relevant flights.

A complete async Playwright function that extracts all the flight data points from a Google Flights card element will look like this:

# scraper.py


async def extract_flight_data(card):
    """Extract individual flight data points from a Google Flights card."""
    data = {
        "price": None,
        "departure_time": None,
        "arrival_time": None,
        "duration": None,
        "stops": None,
        "airline": None,
        "emissions": None
    }
    # Price
    try:
        price_element = await card.query_selector(".YMlIz")
        if price_element:
            data["price"] = (await price_element.inner_text()).strip()
    except:
        pass
    # Departure and arrival times
    try:
        time_elements = await card.query_selector_all("span")
        if len(time_elements) >= 2:
            data["departure_time"] = (await time_elements[0].inner_text()).strip()
            data["arrival_time"] = (await time_elements[1].inner_text()).strip()
    except:
        pass
    # Flight duration
    try:
        duration_element = await card.query_selector(".gvkrdb")
        if duration_element:
            data["duration"] = (await duration_element.inner_text()).strip()
    except:
        pass
    # Number of stops
    try:
        stops_element = await card.query_selector(".EfT7Ae")
        if stops_element:
            stops_text = await stops_element.inner_text()
            data["stops"] = 0 if "Nonstop" in stops_text else int(stops_text.split()[0])
    except:
        pass
    # Airline name
    try:
        airline_element = await card.query_selector(".sSHqwe")
        if airline_element:
            data["airline"] = (await airline_element.inner_text()).strip()
    except:
        pass
    # Emissions (optional)
    try:
        emissions_element = await card.query_selector(".V1iAHe")
        if emissions_element:
            data["emissions"] = (await emissions_element.inner_text()).strip()
    except:
        pass
    return data

# scraper.py


async def extract_flight_data(card):
    """Extract individual flight data points from a Google Flights card."""
    data = {
        "price": None,
        "departure_time": None,
        "arrival_time": None,
        "duration": None,
        "stops": None,
        "airline": None,
        "emissions": None
    }
    # Price
    try:
        price_element = await card.query_selector(".YMlIz")
        if price_element:
            data["price"] = (await price_element.inner_text()).strip()
    except:
        pass
    # Departure and arrival times
    try:
        time_elements = await card.query_selector_all("span")
        if len(time_elements) >= 2:
            data["departure_time"] = (await time_elements[0].inner_text()).strip()
            data["arrival_time"] = (await time_elements[1].inner_text()).strip()
    except:
        pass
    # Flight duration
    try:
        duration_element = await card.query_selector(".gvkrdb")
        if duration_element:
            data["duration"] = (await duration_element.inner_text()).strip()
    except:
        pass
    # Number of stops
    try:
        stops_element = await card.query_selector(".EfT7Ae")
        if stops_element:
            stops_text = await stops_element.inner_text()
            data["stops"] = 0 if "Nonstop" in stops_text else int(stops_text.split()[0])
    except:
        pass
    # Airline name
    try:
        airline_element = await card.query_selector(".sSHqwe")
        if airline_element:
            data["airline"] = (await airline_element.inner_text()).strip()
    except:
        pass
    # Emissions (optional)
    try:
        emissions_element = await card.query_selector(".V1iAHe")
        if emissions_element:
            data["emissions"] = (await emissions_element.inner_text()).strip()
    except:
        pass
    return data