Get the HTML content from the database [closed]

Question

I have a copy of a Drupal 9 database from which I need to download all the pages.
I loaded the database in MySQL Workbench CE and connected to it via Python. There are many tables, but no views or stored procedures. I guess that some of those tables house the content, but I have no idea on how to pull them together to extract webpages.

block_content_body looks promising, but then what?

I assume it is a standard Drupal database. Is there a standard schema?

This what I have tried.

#connect to drupal db - done...
#copy to directory assigned.
#othercode config....

try:
    # Define the tables and columns to extract HTML content from
    tables_columns = [
        ('node_field_data', 'body_value'),
        ('block_content_field_data', 'body_value'),
        ('field_data_body', 'body_value'),
        ('field_data_[custom_field_name]', '[custom_field_column]'),
        ('paragraph__field_[custom_field_name]', '[custom_field_column]'),
        ('views_view', 'display_options')
        # Add additional tables and columns as needed
  

    # Iterate over the tables and columns and export HTML content to separate files
    for table, column in tables_columns:
        output_file = os.path.join(output_dir, f'{table}_{column}.html')
        query = f"SELECT {column} FROM {table}"
        cursor.execute(query)
        rows = cursor.fetchall()

        with open(output_file, 'w', encoding='utf-8') as file:
            for row in rows:
                html_content = row[0]
                file.write(html_content + '\n')

        print(f"HTML content from {table}.{column} exported to: {output_file}")

except mysql.connector.Error as error:
    print(f"Error retrieving data from MySQL: {error}")

finally:
    # Close the database connection
    if connection.is_connected():
        cursor.close()
        connection.close()


I still have to correct the field names but guess these are the html files some how joined together here?

Do you have suggestions, or is this just crazy talk?

Answer 1

Drupal pages are generated in a complex way. Even the content stored in the database is often parsed by an input filter prior to processing. So you will need to use Drupal to generate each page you want a copy of and save that.

But... that's a pain, so there are some automated ways to do it. The downside is that it takes some work to set them up.

If you want to do this repeatedly, you can try the Tome module. This allows you to generate a static site from Drupal 8+ on demand.

Alternately, you could look into setting up Gatsby with Drupal.

Answer 2

The appropriate and documented way to do this is to run the Drupal website on a web server and then create a static archive of the website.

Answer 3

You can use Tome for that.

Tome is a static site generator, and a static storage system for content.

When Tome is enabled, any changes to config, content, or files will be automatically synced to your local filesystem. These exports can be used to fully rebuild the site from scratch, which removes the need for a persistent SQL database or filesystem. When you're ready to push to production, you can use Tome to generate a static HTML version of your site.

Long story short, you can use Drupal in the same way you would use other static site generators like Jekyll or Hugo - everything lives in one repository, and Drupal only runs on your local machine.