Dec 02, 2019

Which Flutter widgets you should learn first...according to SCIENCE.

A couple months ago, I had the question "Would it be possible to code a Flutter app visually without code?" I got my answer and more. But as it goes with big questions, they often lead to more questions.

Ok, cool, you can generate Flutter code with blocks but making a block per widget and implementing all widgets is unsustainable, you'd have to pick a subset. But what subset of widgets would you pick? More to the point, which widgets are used the most in Flutter apps? How would you even figure it out? Can you even?

I put the idea on ice and when I wasn't expecting it came to me in the shower or a dream: ANTLR. Some androids dream of electric sheep, I dream of Backus-Naur form. ANTLR, Another Tool for Language Recognition, is a tool to generate recognizers and create compiler generators for programming languages. It's one of those things you have for a semester in college and likely promptly forget. Or not. I apologize in advance for the small jump into traditional computer science theory, it's worth it, painless, and I'll be quick.

A Bit of Theory

Languages today are generally defined at their most basic level by a grammar. This grammar will define the basic components of a language in terms of what characters, digits, and constructs may be composed to make other constructs. Take this grammar for instance:

grammar Expr;      
prog:  (expr NEWLINE)* ;
expr:  expr ('*'|'/') expr
    |  expr ('+'|'-') expr
    |  INT
    |  '(' expr ')'
    ;
NEWLINE : [\r\n]+ ;
INT     : [0-9]+ ;

It defines a language that defines a program as one or more expressions, each on their own line. Each expression is either an integer, an expression in parens, or an arithmetic expression. The unquoted * and +, Kleene star and plus respectively, are the operators you might recognize from regular expressions to indicate zero or more and one or more.

For a small enough single input, you could parse it with a regular expression. That becomes unwieldy for something like (5 + (34 * 43 + (15-2 * (42-3 / (6 * 2)))).

Now that we have a grammar, one step that must be done before we can work with the code is to send it through a lexer (or tokenizer). It looks at the code character by character and tries to match it to a rule or as a literal. It defines the WHAT that is in your app. Next, the parser takes that set of tokens and tries to apply the parser rules (the lower case ones), arranging the code into an Abstract Syntax Tree, uncovering any possible syntax errors. The parser is what makes sure you aren't a monkey typing at a keyboard and it all makes sense.

During this tree composition stage, you can write a listener to inform you when a rule has been entered or exited. Modern programming languages may use this step to in a parser to generate code for whatever syntactic sugar shortcut or @Whatever annotation, your language may have. This is also our stop to jump back into Flutter-ville.

Back to Flutter-ville

Deep in the dark recesses of the Dart SDK is a ANTLR grammar for the language. ANTLR's TestRig utility[the images above and below] allows you to see the AST for a given bit of code. A bit of exploratory poking about in files gave me an idea of where to find things. The parse trees for a Dart files are understandably more complex than Expr.

The names of the widgets could be found by writing a enterTypeWithParameters rule for my listener and iterating over all the Material widget files. Finding usage of widgets in user code was a bit trickier, needing to listen for identifiers and named argument lists.

class ClassListener(var list: IndexTreeList<Any?>) : Dart2BaseListener() {
    override fun enterTypeWithParameters(ctx: Dart2Parser.TypeWithParametersContext?) {
        val identifier = ctx?.text
        if (identifier != null) {
            list.add(mapOf<String, String>("id" to identifier))
            println(identifier)
        }
    }
}

Intaking all this data left me with a rather messy dataset that would need some cleaning to remove false positives(user created classes) and remove some internal classes that are never meant to be used by end-users. Krangl, a Kotlin library for data analysis similar to dplyr from R, helped me massage things into place. The source files were pulled from some open source Flutter apps and some Google samples.

Before I get to the results, let's do some predictions and fun facts. My prediction going in, because what good experiment lacks a hypothesis, was the following to be the top 5 widgets in no specific order:

• Text

• Row

• Column

• Container

• Icon

The largest file was 101KB. Its' parse tree was so big and wide, a whooping 94573 x 4326 pixels that it was illegible and crashed several image viewer apps. Whereas my sample data set could process all files at once, for the big data set, I had to batch process them.

The Results

The top 10 widget frequencies were :

1. Text

2. Container

3. Padding

4. Column

5. Icon

6. Row

7. SizedBox

8. Center

9. Expanded

10. Scaffold

Padding was an unexpected but understandable surprise. I also pulled a top 20. The big surprise on that list was seeing InkWell edge out ListView which I had pegged for a top 10 or just outside.

I generated some graphs on the widgets in the top ten to show which properties were used most often. No real surprises there for the most part. Apologies for one or two of the labels being so long that they become illegible.

Last fun note, the top 20 widgets are used 78.21% of the time in my data set. Yay for the Pareto principle(80/20 rule).

Full res versions of the generated charts here: https://github.com/jwill/flutter-analysis/tree/master/pngs

My code is on Github: https://github.com/jwill/flutter-analysis

Have fun, explore, and feel free to tell me what I missed.

Working on document steps to reproduce a bit better but you should be able to put pop it into IDEA/Android Studio, run generateGrammarSource then DartTest and be up and running.