CodeWeb: Web-like, Unified Code/Documentation Experience in IDEs

Source code files should be more like WYSIWYG-editable web pages. IDEs should be more like web browsers.

15 min readMay 21, 2019

In the post about semi-functions, I mentioned the idea of going beyond the assumption that source code is organized in flat plain-text files which should allow programmers to work more effectively with code. In this post, I elaborate on that and suggest more similar ideas on the IDEs’ front adding up to a vision of the development experience (which is mostly navigation and reading, not writing) like browsing between hierarchically organized rich hypermedia documents which combine code, documentation, and comments. I’ll call it the CodeWeb vision (interface, experience).

I’ve put most of the CodeWeb ideas into the context of IDEs on the IntelliJ Platform because I’m more familiar with it. These ideas should be likewise applicable in other traditional IDEs such as Eclipse, Visual Studio, VS Code, and Xcode.

CodeWeb vision is not a “reimagine everything” kind of thing but rather a roadmap to improve the existing experiences to incentivize developers to

Write more comments explaining the behavior and the “why” of the code;
Create design documents and diagrams which don’t fall out of sync with the code;
Make the codebases more easily navigable via lots of hyperlinks.

Many of the ideas presented in this post date almost 40 years back to literate programming by Donald Knuth. It feels like now is the time when IDEs have grown in capability to bridge the gap and provide a unified, instant feedback experience for understanding, designing, and coding the software.

WYSIWYG comment editing with headers, links, and accents

In the majority of projects, documentation comments are read predominantly right from the IDE interface itself rather than from websites with generated documentation. The only exception I can think of is documentation in popular library projects and SDKs which may be consumed more often in the processed form on websites (such as Guava documentation), but even that is not guaranteed because when working with a library developers might prefer to go to a library’s symbol and read its documentation within IDE, as source. Hence showing readers the raw formatting is as reasonable as showing people HTML source instead of rendered HTML in browsers.

Additionally, very long URLs (as well as links to members in the code) may not fit the maximum line width imposed by the coding style. To please rigid coding style checking tools the URLs should be split into multiple parts, making them harder to navigate through.

An example of the “long URL” issue from Apache Druid

Editing of code comments should be more like modern WYSIWYG editing experience which never exposes the “raw” formatting (which is HTML or Markdown in the case of WYSIWYG like the Medium’s post editor and the documentation comment formatting (like Javadoc formatting) in the case of code editors).

In other words, when editing, for example, a Javadoc comment, a developer should never see {@link ...} , {@code ...}, <i>...</i>, etc. A developer just highlights a part of the text that they want to make a link, or bold, or italic, and the code editor shows a floating toolbox to achieve that effect, like in Medium:

When editing is completed, the comment just appears as a link, or in a monospaced font, or in italics, etc.

Block comments (documentation comments and comments that span the line of code until the end, rather than comments inside statements like x = y + /* I'm a comment inside a statement! */ z) should be rendered in a non-monospaced font by default. Some parts of them can be formatted as monospaced to distinguish references to variables and members from normal text, or when a comment aligns with the surrounding code, for example:

//`I want to point to ↓ d`
someFunction(a, b, c, d);

Another case for using a monospaced font in the comments is preserving old ASCII art pictures or diagrams, although CodeWeb interface should support both natively: see the following two sections in this post.

The problem of WYSIWYG comment editing is recognized for the IntelliJ platform. Artsiom Chapialiou even created a plugin that solves this problem for Java and Javadoc formatting. However, probably only a native implementation in the IDE can provide a really smooth experience. The plugin also doesn’t change the presentation of ordinary, non-Javadoc comments, while there should be no distinction in formatting capabilities between “documentation” comments and comments inside functions.

Note that there is an important distinction from some modern implementations of literate programming (Literate CoffeeScript, Jupyter Notebooks): I don’t propose to reverse the priorities and make the comments (“literature”) more important than the code. The comments should still be secondary (at least by default), the color of their font should have less contrast with the background editor’s color than the color(s) of the code constructs.

Tables in comments

I really like the popover interface of the Medium’s editor, but I think it lacks one important feature which is available in some older WYSIWYG interfaces: visual editing of tables (including merging and splitting table cells). This is a not very often, but a regularly needed feature in the API documentation, especially in class-level and subsystem overview documents (for example, see a table in the Java’s BlockingQueue documentation). The HTML formatting like <table> <tr> <td> ... </td> <td> ... </td> ... is very tedious to create by hand and is incomprehensible in the raw form.

Module, class, and control/data flow diagrams in documentation

I think that UML diagrams of class relationships and control or data flow diagrams are invaluable for quickly building an understanding of the architecture or the concurrent execution model of software subsystems. (See a related item in the code review checklist for concurrency in Java.)

However, developers very rarely embed diagrams into the documentation. I think there are several reasons for that:

If a diagram is created using a vector graphics editor, it’s outside of the “project’s control” and the source file may be in a proprietary format (such as .vsd) or even unavailable (such as in case of online tools like Google Drawings). Only the author of the original diagram may be able to adjust it to match the changes in the software. If that developer leaves the project or diverts to other tasks the diagram falls out of sync with the code.
Diagrams are not visible in the documentation when viewed from the IDE (which is the primary way of consuming documentation, as noted above). Firstly, it diminishes the value of creating diagrams in the first place, because developers won’t see pictures and refresh their mental models as they “pass by” the code. Secondly, if a diagram is a link to an external image in the documentation it’s both more likely that developers will forget to update it as they update the code, and that they won’t notice the inconsistency for a long time (because the experienced developers may not check diagrams often), while the developers who learn the codebase may be confused by the inconsistency to the degree that the diagram will make more harm than good for them. (Currently, developers sometimes resort to ASCII art to create diagrams visible in the documentation, but the expression power of ASCII art diagrams is very limited.)
The toil of updating the diagram every time some class, or a function, or a field is renamed, or added, or removed may intimidate the developers. Let alone that the developers who rename or change members should somehow become aware that there is some diagram somewhere in the documentation that includes the changed or removed entity or should include the newly added entity.

The third argument, I think, makes it an absolute necessity that class UML and module dependency diagrams update themselves automatically with the changes in the structure of the code. Here are the other features that should incentivize developers to enrich documentation with diagrams:

Zoomable diagrams should be visible, embedded right in the documentation when viewed from the IDE.
Apart from elements in the diagram that represent an entity in the code (a module, a class, a function, a field), a developer can add arbitrary other elements to the diagram, such as text boxes with additional commentary, extra arrows between arbitrary elements, and graphics. If there is an extra arrow or a text box attached to an entity from the code which is removed, the diagram becomes “broken” and IDE reports this as an error.
Diagrams should be WYSIWYG-editable within IDE, right in the documents where they are embedded or at most one click away.
Whenever a diagram is changed, either manually or due to a structural change in the code, a file in an open format (such as .svg) which backs the diagram is updated synchronously. The file is located in the project’s resources directory (such as src/main/resources in Java projects) and is automatically included in the generated HTML documentation (such as Javadoc) without any extra effort required from the developers.

It seems that both IntelliJ and Visual Studio can only visualize dependency and class diagrams in separate windows, not embedded in the documentation. At least in IntelliJ, diagrams seem to be ephemeral: unless exported immediately to an image file, all changes to the layout and added notes are lost when a diagram is closed and then opened again.

IntelliJ and Visual Studio only allow adding comments to diagrams, but not arbitrary extra structures: boxes and arrows of different shapes. That would enable creating control flow and data flow diagrams in an arbitrary format. These diagrams should be able to contain elements that are “hard-linked” to some entities in code (functions, classes, and fields) as well as boxes corresponding to classes in UML diagrams. For example, a control flow diagram may need to refer to a field containing an executor to identify that some logic is performed in that executor. Data flow diagrams may refer to functions as data processing steps.

Images in comments

IDEs should render images in comments. The primary use case for images in the documentation, I think, is using them as a quick alternative to proper diagrams (see the previous section): while designing the system (or teaching the design of an existing system to new people), developers may draw diagrams on a whiteboard. Then these diagrams can be quickly added to the project’s artifacts by taking a photo of the whiteboard and inserting it into the documentation.

Here is a relevant feature request for IntelliJ.

Support for inserting images in comments has been added recently to VS Code:

Unrestricted links

Ordinary, non-documentation comments should support links to classes, members, and external resources just as well as documentation comments. Links to classes and members in ordinary comments, in documentation comments to private members, or anywhere (if configured) should be allowed to point anywhere in the project, rather than just to classes and members that can be referenced (that is, accessible) from the respective code. The latter is a limitation imposed by how documentation generation tools (such as javadoc) work, which is irrelevant when documentation is not, in fact, generated and the documentation comments are more often consumed from the IDE itself, as pointed out above.

Permitting developers to link anywhere in the project from any other place would allow developers to make navigation in the codebases easier by connecting elements via links which can be followed in both ways, across dependency/dependent module boundaries in multi-module projects.

Relevant issues for IntelliJ: 1, 2, 3.

Wiki-style links to labels and long comments

Sometimes, a piece of code, an expression, or a statement warrants a non-trivial comment explaining why the code looks exactly how it does, or explaining some involved tradeoffs. That piece of code may also be repeated in many places and can’t be extracted into a function because it’s too low-level or because it has multiple outputs and the programming language (such as Java) doesn’t support multi-return functions efficiently.

In this situation, the non-trivial comment should either be repeated at each occurrence of the piece of code in question, which exacerbates repetition (the code itself may be a single expression or a single statement, while the explaining comment can be 10 lines long), or some type of light-weight linking to specific comments should be supported. The convention that I came up with in my projects is starting a comment with a “header” ending with a colon:

// Reading dataSources once in a local variable: the field must not
// be read multiple times within a single method because it may be
// nulled out concurrently in stop(), so the second use might result
// in NPE even if it was non-null during the check.
val dataSourcesSnapshot = this.dataSources;
if (dataSourcesSnapshot != null) {
  ...
}

Then, the whole comment is referenced as [Comment's header] wherever the piece of code or the idiom appears (compare with MediaWiki’s double square brackets):

// [Reading dataSources once in a local variable]
val dataSourcesSnapshot = this.dataSources;
if (dataSourcesSnapshot != null) {
  ...
}

This referencing scheme generalizes to block and statement labels:

fun foo() {
  ... a lot of code
  myLoop@ for (...) {  }
}fun bar() {
  // See [myLoop] in foo().
}

So such wiki-style links can be used whenever a specific place inside a large function should be referenced from elsewhere. The remaining problem is that there is no support for such references in IDEs: currently, to navigate through them one needs to copy a wiki link’s text into the clipboard and invoke the search interface. Also, IDEs should track when such references are broken (for example, when the referenced “header” is removed along with the whole function or the comment).

Support for wiki-style links to labels and long comments would allow improving the navigability of the codebases. Developers would be able to reduce repetition in comments without sacrificing their completeness.

Long comments: fade out, “see more”

Sometimes, a single line or a block of code warrants a really long comment with an explanation of the tradeoffs involved, historical background, discussion of why some other ways of writing that code (which may seem better at first) don’t work, and so on. Such comments may span dozens of lines.

However, there are two issues with long explanatory comments:

They greatly increase the vertical size of the containing function. Long comments “break the code flow”.
They draw too much attention to themselves. A developer may have already seen the comment (or have written it) and want to focus on the other aspects of the function, but the large comment grabs the attention each time developer browses the function’s code.

These considerations put pressure on developers to keep their comments short, and many things in code end up being discussed in comments much less thoroughly than they ought to.

IDEs can help to release this tension via two features:

Automatic fade out (like here) of comment’s text after, say, the third line of the comment. Number of lines before fading out can be configurable.
A developer can put a “cut” manually in their comment, for example, after the first sentence in the comment, and the editor interface presents “see more…” / “see less…” links (or just “+” and “−” buttons, only right in the comment’s text rather than on the left vertical bar of the editor’s pane, like normal code folding; it’s better when developers need to drag the cursor less). The “cut” can be added and removed from a long comment in the same visual style as discussed in the “WYSIWYG comment editing” section above.

In IntelliJ-based IDEs, it’s already possible to fold multi-line comments. However, there are two problems with the existing comment-folding feature:

Comment’s text is folded completely, to just three dots, without a hint of what the comment is about.
Folding is not eager, that is, expanded comments are not collapsed back automatically when they move out of the reader’s sight. Readers forget to collapse them manually after reading (and, well, this is also additional efforts), and the code eventually moves to the “most comments are unfolded” state.

Variable expansions registry

In the Go programming language, there is a convention to use single-letter variable names which some programmers don’t feel comfortable with. IDEs may support a registry of variable name expansions. Each variable, say, n may be associated with an expansion, for example, “numBytes”. In plain-text (fallback) form, expansions can just be comments alongside the variables’ declarations, for example:

var n: Int = 0 // numBytes
// or
fun copy(from: Buffer, to: Buffer, n: Int /* numBytes */) {
 ...
}

IDEs may do the following:

When a variable doesn’t have an expansion attached yet, a special action invokes a search over other variables with the same name and type that already have expansion in the codebase and suggests to attach the same expansion to the variable. For example, if there is a function write(f: File, b: Buffer, n: Int), invoking the action with the cursor at n suggests to attach expansion “numBytes” because it is attached to the parameter n of the copy() function. If there are several variables in the codebase with the same name and type that have different expansions (such as numBytes and numItems), the action suggests choosing among them (or to create a new expansion, of course).
Optionally, hide expansion comments in a form similar to <abbr> tag’s presentation on web pages.
Optionally, show the expansion when the cursor pointer is over any use of a variable for several seconds, that is, exactly how <abbr> tag works in the Web, but without dotted underlining of each use of the variable.

Completely hidden comments

Combining the ideas from the two previous sections, CodeWeb interface can support comments to statements that are completely hidden by default. Only underlining (like produced by HTML’s <abbr> tag in the Web) or an asterisk (another idea borrowed from Medium, where asterisks identify comments from post’s reviewers) to the statement would reveal that there is an associated comment which may be shown when the reader moves the cursor over the asterisk.

This feature may be useful, for example, in the same case as explained in the “Wiki-style links to labels and long comments” section above in this post: when there are repetitive statements or expressions in code warranting comments, yet as soon as the reader familiarizes with the problem such comments become a distraction.

In CodeWeb interface, one of the examples above may look like this:

val dataSourcesSnapshot = this.dataSources; *
if (dataSourcesSnapshot != null) {
  ...
}

And when a reader moves the cursor over the asterisk (the asterisk should be smaller and in a low-contrast color with the background so that it doesn’t look as a part of the code) the hidden comment [Reading dataSources once in a local variable] floats up.

In the plain-text (fallback) form, completely hidden comments may start with a special prefix, for example, “Side note”:

// Side note: [Reading dataSources once in a local variable]
val dataSourcesSnapshot = this.dataSources;
if (dataSourcesSnapshot != null) {
  ...
}

Code style as website’s CSS and as user CSS

To reduce the heat of code style debates, IDEs should allow programmers to configure different code style aspects locally as they prefer:

Indentation style.
Maximum line width and the associated style of function arguments wrapping and other line-wrapping style matters. In fact, IDEs should adapt to the current width of the editor pane, just like web browsers change the layout of adaptive web pages for different window sizes.
Function arrangement: depth-first or breadth-first. After publishing the post about semi-functions, I’ve realized that they can be supported entirely within IDEs as a special, “tree” function arrangement mode rather than on the level of the syntax and semantics of a programming language.

To avoid unnecessary churn in commits and ensure consistency in fallback plain-text code representation, IDEs should be able to internally reformat code according to some code style settings shared among the developers of a project, while not changing the presentation in the editor. In the Web analogy, these two different code styles correspond to CSS of a website and CSS that users can apply to change the website’s appearance.

Note: CodeWeb vision described in this post has nothing to do with the web IDE (online IDE, cloud IDE) concept (see projects such as Eclipse Che and Codeanywhere). Web IDEs may or may not implement some parts of CodeWeb, as well as traditional IDEs. In modern operating systems which support custom URL handlers and given that technologies like Electron are available, web IDEs have little or no headstart over traditional IDEs for implementing CodeWeb ideas.

Acknowledgments

First of all, I would like to thank awesome IDE developers who created tools that are already incredibly powerful and have put the CodeWeb vision described in this post into the adjacent possible.

Thanks to IDE enthusiasts such as Artsiom Chapialiou who spend their time creating plugins to explore ideas about the development interface.

Thanks to Steve Tockey who described the idea of “hierarchically organized rich hypermedia documents” in his great Software Design course (see the post with the final passage from the course) which helped me to solidify the CodeWeb vision by putting individual ideas into a coherent perspective.