Complex pages and webpack/Babel configuration in ACT

ACT is a starter to create blog/documentation sites. It provides advanced, non-trivial building blocks, including pages generated by gatsby-plugin-combine. The focus is on flexibility and progressive enhancements, made easy through multiple GraphQL queries and clear-cut data processing. And it comes with a popular Ant Design library, very helpful when creating complex React components, and MDX (markdown with embedded JSX components, see also our MDX intro) to handle chunks of structured text. The blog itself has two main regular pages: blog posts and snippets (short excerpts), which could be useful whenever there is a need to describe and summarize/quickly reference some information. As ACT relies on a modern web development ecosystem, it is impossible to cover all the aspects which could be useful when working with this starter. Instead, this article gives overview of the project structure, gatsby-plugin-combine, and advance Babel configuration options. For a higher level overview you may read an ACT intro.

Complex pages using gatsby-plugin-combine

Gatsby pages are regular React components which receive some page context (a JSON object). A page component may also define a GraphQL query. This greatly increases flexibility, as one can declaratively query and process data of practically unlimited complexity. The only, but important, caveat is that a Gatsby node graph should provide a sufficiently convenient data representation. And in practice, indeed, when working with Gatsby, one has to face some challenges.

Firstly, after page GraphQL query is executed, complex data preprocessing is still often required, in which case code may beg for explicit extraction of data processing logic. In general, separation of data processing from UI code may encourage a better structured, less coupled with Gatsby, and more testable code. And, as an additional bonus, during explicit data processing, information always could be easily saved or logged for later inspection or reuse.

Secondly, for complex pages, it is often convenient to think that every page consists of multiple, logically independent parts. When code is structured accordingly, it clarifies GraphQL queries and data processing logic. Along the same lines, lodash templates, for example, can serve as an attractive, flexible tool to mix and match logical page elements.

And finally, additional GraphQL queries or access to the data outside GraphQL could be desirable. Shaping GraphQL to the needs of every website page could be too time consuming, to be practical. Developers may prefer to leave Gatsby nodes intact, but, instead, read additional data directly from external data sources, or query GraphQL multiple times.

All the complications mentioned could be addressed using the power of the Gatsby API, which makes every aspect of page construction transparent and open to customization. ACT doesn't use the Gatsby API directly, but, instead, relies on gatsby-plugin-combine, which provides a convenient abstraction and API to work with every page as a collection of multiple GraphQL queries/data processing functions.

For example, it is convenient to treat snippets as chunks of text plus elaborate navigation (a sidebar menu). Chunks of text, possibly with some embedded components (code excerpts, additional controls, charting elements, etc), could be represented using MDX files. A sidebar menu, on the other hand, requires information about all snippets in the collection and additional configuration. Thus, navigation information is obtained once for all snippets (using a separate GraphQL query) and is saved in a JSON file to be imported later by all snippet pages. To combine MDX, JSON, and React component files, lodash templates come in handy.

// a helper module for snippet pages

// a function to query a layout configuration and all snippets
function layoutAndMenuQuery(){
    return `
    query LayoutQuery($parent: String, $parent_re: String) {
        layout:snippetsYaml(fields:{slug:{eq: $parent}}){
            menu {
                text
                icon
            }
        }

        snippets:allMdx(filter:{
            fields:{slug: {regex:$parent_re}}
        })
        {
            edges{          
                node {
                    frontmatter{
                        menu
                    }
                }
            }
        }
    }
    `   
}

// a function to query content frontmatter
function contentQuery(){
    return `
    query SnippetQuery($id: String) {
        mdx(id: { eq: $id }) {
            frontmatter {
                title
                keywords
                menu
            }    
        }
    }
    `
}

// preprocess menu data (this function is executed once per snippet collection)
function processMenu({layout, snippets}, {writeData}){
    ......
    const fileName = writeData("../layout.json", {menu, items}) // save layout data

    return {layout:fileName};
}

// preprocess data for content
function processContent({mdx}){
    return {frontmatter: mdx.frontmatter}
}

// finish page creation
function combine({layout, frontmatter}, {generatePage, createPage}){
    generatePage({layout}); // generate a page using a selected lodash template
    createPage({frontmatter}); // add a new page
}

module.exports = {
    entities : { // queries/data processing for distinct page elements
        layout:{
            scope: "parent", // process data only once for all snippets in a collection
            query: layoutAndMenuQuery,
            data: processMenu
        },

        content:{
            query: contentQuery,
            data: processContent
        }
    },
    combine // finish page creation
}

Posts, on the other hand, are simpler and could be implemented using standard Gatsby tools. Still, in ACT, they are implemented similarly to snippets: a generated page wraps MDX inside a React component. This leaves more knobs to play with and makes the implementation more flexible/open to progressive enhancements, when and if further enhancements become desirable. But even if no further enhancements is expected, this approach has certain advantages: shortness of code and simplicity are traded for code decoupling and explicit page representation - arguably a reasonable tradeoff.

The index of blog posts is a unique page and hence located in the src/content/pages directory. It is also quite a complex page, which may grow in complexity even further, so gatsby-plugin-combine is used again.

Configuring Babel and Webpack

As our blog aims to provide reach interactive user experience, we need complex page components, and, to make complex React components easier to develop, we need a powerful and diverse React library. A popular Ant Design library fits the above description well enough, so it was a natural choice. But it is not enough to install the library itself (and a gatsby less plugin to compile stylesheets) to start working productively - a convenient JavaScript import style is also important. And here the custom Babel configuration (in a .babelrc file) and babel-plugin-import come to the rescue:

// a custom Babel configuration file (.babelrc)
{
    "presets": [
        [ //first specify a gatsby preset
            "babel-preset-gatsby",
            {
                "targets": { // old browsers support
                    "browsers": [">0.25%", "not dead"],
                },
            },
        ],
    ],
    "plugins":[ // short import syntax for Ant Design and project components
        ["import", { "libraryName": "antd", "libraryDirectory": "lib", "style": true}, "ant"],
        ["import", { "libraryName": "actb", "libraryDirectory": "src/components"}, "actb"],
    ]
}

// succinct import for antd components
import {Button, Icon} from "antd";
// and for project components (assuming an "actb" alias is defined)
import {Mdx, Logo} from "actb";

To import project components as "actb" and simplify other imports, webpack aliases must be defined:

// a Gatsby API hook function to customize webpack configuration (here, to add aliases) in gatsby-node.js
function onCreateWebpackConfig ({actions}) => {
    actions.setWebpackConfig({
        resolve: {
            alias: {
                "components": path.resolve("./src/components"),
                "content": path.resolve("./src/content"),
                "pages": path.resolve("./src/pages"),
                "generated": path.resolve("./src/generated"),
                "actb": path.resolve("./"),                                        
            }
        }
    })
}

The style of imports is no doubt a matter of taste and preferences, and one may prefer to use traditional relative imports, but if you do not mind aliases, all necessary infrastructure is already in place. Webpack configuration adjustments in Gatsby projects conventionally happen in a onCreateWebpackConfig hook in gatsby-node.js. To reduce clutter and facilitate later code reuse, this and other common configuration operations in ACT are moved to the functions defined in the gatsby directory.

Project directory structure

An ACT project directory structure follows usual Gatsby conventions. For example, files in src/pages are ordinary Gatsby pages. The src/components directory, as usually, contains React components. In addition, there is a number of directories specific to ACT.

Resources for pages are stored in src/content subdirectories:

• config - common configuration files (main.yaml and other).
• images - a logo and other images.
• pages - resources for unique complex pages (the main page, index of posts, etc).
• posts - MDX (markdown) files for regular blog posts.
• snippets - collections of snippets (subdirectories and corresponding configuration files). Every subdirectory is a single collection of snippets (MDX files). Every collection has a configuration file with the same file name but an *.yaml extension.

ACT also uses the data directory, where helper modules of gatsby-plugin-combine can save data. There is also a number of src subdirectories for gatsby-plugin-combine:

• generated - the place where pages, generated using lodash templates are saved. The path of a page relative to src/generated corresponds to the slug of the page.
• helpers - node.js modules orchestrating page construction (a helper plugin option).
• templates - lodash templates (a template plugin option).

ACT project 
│
└───src
│   │
│   └───components // project components 
│   │
│   └───content // resources used to construct pages
│   │   │   
│   │   └───config // common configuration files (*.yaml, *.json, etc)
│   │   │   
│   │   └───images // a logo and other images
│   │   │
│   │   └───pages  // unique complex pages (the main page, index of posts, etc)
│   │   │
│   │   └───posts  // blog posts (markdown files)
│   │   │
│   │   └───snippets  // snippets
│   │       │
│   │       └───main  // a <main> collection of snippets (a directory with markdown files)
│   │       │
│   │       └───main.yaml  // a configuration file (corresponding to the <main> collection)
│   │
│   └───generated // pages generated using lodash templates (gatsby-plugin-combine)
│   │
│   └───helpers // helper modules (gatsby-plugin-combine)
│   │
│   └───pages // standard Gatsby pages
│   │
│   └───templates // lodash templates (gatsby-plugin-combine)
│   
└───data // data files saved by helper modules (gatsby-plugin-combine)