---
title: Using Playwright
sidebar_label: Using Playwright
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import CodeBlock from '@theme/CodeBlock';

In this guide, you'll learn how to use [Playwright](https://playwright.dev) for web scraping in your Apify Actors.

## Introduction

[Playwright](https://playwright.dev) is a tool for web automation and testing that can also be used for web scraping.
It allows you to control a web browser programmatically and interact with web pages just as a human would.

Some of the key features of Playwright for web scraping include:

- **Cross-browser support** - Playwright supports the latest versions of major browsers like Chrome, Firefox, and Safari,
so you can choose the one that suits your needs the best.
- **Headless mode** - Playwright can run in headless mode,
meaning that the browser window is not visible on your screen while it is scraping,
which can be useful for running scraping tasks in the background or in containers without a display.
- **Powerful selectors** - Playwright provides a variety of powerful selectors that allow you to target specific elements on a web page,
including CSS selectors, XPath, and text matching.
- **Emulation of user interactions** - Playwright allows you to emulate user interactions like clicking, scrolling, filling out forms,
and even typing in text, which can be useful for scraping websites that have dynamic content or require user input.

To create Actors which use Playwright, start from the [Playwright & Python](https://apify.com/templates?category=python) Actor template.

On the Apify platform, the Actor will already have Playwright and the necessary browsers preinstalled in its Docker image,
including the tools and setup necessary to run browsers in headful mode.

When running the Actor locally, you'll need to finish the Playwright setup yourself before you can run the actor.

<Tabs groupId="operating-systems">
    <TabItem value="unix" label="Linux / macOS" default>
        <CodeBlock language="bash">{
`source .venv/bin/activate
playwright install --with-deps`
        }</CodeBlock>
    </TabItem>
    <TabItem value="win" label="Windows">
        <CodeBlock language="powershell">{
`.venv\\Scripts\\activate
playwright install --with-deps`
        }</CodeBlock>
    </TabItem>
</Tabs>

## Example Actor

This is a simple Actor that recursively scrapes titles from all linked websites,
up to a maximum depth, starting from URLs in the Actor input.

It uses Playwright to open the pages in an automated Chrome browser,
and to extract the title and anchor elements after the pages load.

```python title="src/main.py"
from urllib.parse import urljoin

from apify import Actor
from playwright.async_api import async_playwright


async def main():
    async with Actor:
        # Read the Actor input
        actor_input = await Actor.get_input() or {}
        start_urls = actor_input.get('start_urls', [{ 'url': 'https://apify.com' }])
        max_depth = actor_input.get('max_depth', 1)

        if not start_urls:
            Actor.log.info('No start URLs specified in actor input, exiting...')
            await Actor.exit()

        # Enqueue the starting URLs in the default request queue
        default_queue = await Actor.open_request_queue()
        for start_url in start_urls:
            url = start_url.get('url')
            Actor.log.info(f'Enqueuing {url} ...')
            await default_queue.add_request({ 'url': url, 'userData': { 'depth': 0 }})

        # Launch Playwright an open a new browser context
        Actor.log.info('Launching Playwright...')
        async with async_playwright() as playwright:
            browser = await playwright.chromium.launch(headless=Actor.config.headless)
            context = await browser.new_context()

            # Process the requests in the queue one by one
            while request := await default_queue.fetch_next_request():
                url = request['url']
                depth = request['userData']['depth']
                Actor.log.info(f'Scraping {url} ...')

                try:
                    # Open the URL in a new Playwright page
                    page = await context.new_page()
                    await page.goto(url)

                    # If we haven't reached the max depth,
                    # look for nested links and enqueue their targets
                    if depth < max_depth:
                        for link in await page.locator('a').all():
                            link_href = await link.get_attribute('href')
                            link_url = urljoin(url, link_href)
                            if link_url.startswith(('http://', 'https://')):
                                Actor.log.info(f'Enqueuing {link_url} ...')
                                await default_queue.add_request({
                                    'url': link_url,
                                    'userData': {'depth': depth + 1 },
                                })

                    # Push the title of the page into the default dataset
                    title = await page.title()
                    await Actor.push_data({ 'url': url, 'title': title })
                except:
                    Actor.log.exception(f'Cannot extract data from {url}.')
                finally:
                    await page.close()
                    await default_queue.mark_request_as_handled(request)
```

## Conclusion

In this guide, you learned how to create Actors that use Playwright to scrape websites. Playwright is a powerful tool that can be used to manage browser instances and scrape websites that require JavaScript execution. See the [Actor templates](https://apify.com/templates?category=python) to get started with your own scraping tasks. If you have questions or need assistance, feel free to reach out on our [GitHub](https://github.com/apify/apify-sdk-python) or join our [Discord community](https://discord.com/invite/jyEM2PRvMU). Happy scraping!

## Additional resources

- [Apify templates: Playwright + Chrome](https://apify.com/templates/python-playwright)
- [Playwright: Official documentation](https://playwright.dev/python/)
