# Livedoc Incubator Documentation

Livedoc seamlessly integrates source-code statistics provided by Code Recommenders into the HTML output of Javadoc via a new Doclet and several Providers using the headless API of Code Recommenders 2.0 in an OSGi environment.

Additionally, integration with online services like bugtrackers or mailing lists is planned.

## Disclaimer

The Livedoc plugin (respectively the standalone command-line application) is an incubator project developed as part of the Google Summer of Code. Note that it clearly has some rough edges. It’s not production ready yet.

## Latest Online Demo

Sample output of Livedoc can be viewed here: http://download.codetrails.com/livedoc/published/latest/

Disclaimer: The online demo may include third party providers that do not come with Livedoc.

## Release Planning

Version | Release Date | Features
--- | --- | ---
0.7 | 2013-12-21 | "Call Also" Provider, Recursive subpackages
0.8 | 2014-04-07 | New Provider API, Package Summary View, "Method Self Call" Provider
0.9 | 2014-05-14 | "Class Self Call" Provider, Better model resolution, Debug mode
0.10 | 2014-06-11 | Diff. source/model artifact coordinates, Google Analytics Provider
0.11 | 2014-07-09 (←- Current) | JDK 8 Compatibility, DocLint of JDK 8 Javadoc
0.12 | (August 2014) | Interconnected web of Javadoc documentation for different APIs

## System requirements

Livedoc needs a `Java 8 JDK` installed and is not compatible to prior versions of Java.

Since the release of version 0.11, Livedoc is Java 8 compatible and makes use of the extended JDK 8 Javadoc API. This allows us to support the DocLint feature of Javadoc (see below) and clears the way for multiple Javadoc runs on whole source repositories with just one command line call. Furthermore, JDK 8 Javadoc fixes some bugs, which were sometimes critical to Javadoc runs.

However, since we cannot afford a simultaneous release plan, Javadoc is `not compatible with JDK 7` any more, thus you `need` an installed JDK 8 (and not just a JRE!) to use Livedoc.

## Installation

Livedoc comes as an Eclipse product and can be downloaded from milestones product download site.

## Usage Guide

For a quick start, here is an out-of-the-box working command line for Livedoc:

    ./livedoc org.eclipse.jgit:org.eclipse.jgit:3.4.1.201406201815-r -d ./output -highlight

Livedoc will start generating Javadoc for the `org.eclipse.jgit` package with annotated Code Recommenders statistics, stored in the folder `./output`.

This is possible because the `org.eclipse.jgit` coordinate above is both legal in the Maven Central repository (http://repo1.maven.org/maven2/) as well as in the Code Recommenders model repository (http://download.eclipse.org/recommenders/models/luna/)

For a usage synopsis, just run `./livedoc` without any parameters.

### In Detail

Livedoc requires three inputs from you:

1. A Maven repository for source files (packaging: `jar`, classifier: `sources`)

2. A Maven repository for Code Recommenders models

3. An artifact coordinate identifying the JAR file you wish to create Livedoc documentation for

#### Source Repository

The sources JAR files need to be available in a maven. This repository can reside locally on your harddisk or remotely, for instance on a Nexus repository.

A basic testing repository is available from Codetrails. It provides a source code repository with some artifacts from Eclipse projects for testing purposes (you can look up the contents with your internet browser): http://download.codetrails.com/livedoc/sources/

If you wish to use a local repository, you can deploy existing JAR files using the `mvn deploy` command. For example, to deploy your JRE jar to a local Maven repository, just grab your JRE’s source archive (`src.zip` in your JDK installation folder for Oracle JDKs) and type the following (for Java version 1.7):

    mvn deploy:deploy-file -Durl=file:/some/folder/repo -Dfile=src.zip -DgroupId=jre -DartifactId=jre -Dversion=1.7.0 -Dpackaging=jar -Dclassifier=sources

After that, you can use `/some/folder/repo` as your source code repository for Livedoc (on JRE runs).

#### Model Repository

The location of the Eclipse Code Recommenders models (for Eclipse Luna) is http://download.eclipse.org/recommenders/models/luna-m3.

For a list of all available Eclipse releases, go to http://download.eclipse.org/recommenders/models and click on Show Directory Contents.

If you want to livedoc your own source code, you’ll have to mine models for it first. Take a look at http://www.codetrails.com/ctrlflow for further information.

#### Artifact Coordinate

Livedoc uses the maven artifact coordinate notation:

`GROUP_ID:ARTIFACT_ID:VERSION`

You can get the `GROUP_ID` and `ARTIFACT_ID` from your Maven repository. Note that on `VERSION`, you’ll have to always provide the ‘full’ version (x.y.z), e.g. for JRE 1.7 its 1.7.0.

#### Command line

Putting these informations together, the Livedoc command line has the following syntax:

    ./livedoc GROUP_ID:ARTIFACT_ID:VERSION [-sr URL] [-mr URL ...] [-Xdoclint (all|none|[-]<group>) ...] [-Xresume] [-d DIRECTORY] [-debugging] [-encoding ENCODING] [-highlight] [-jar] [-mc GROUP_ID:ARTIFACT_ID:VERSION] [-noDependencies] [-p PROVIDER_ID [PROVIDER_ARGUMENTS] ...] [-splitIndex] [-subpackages "org.eclipse org.apache ..."] [-transitive] [-u URL] [-verbose]

with the following options:

Parameter | Description
--- | ---
GROUP_ID:ARTIFACT_ID:VERSION | The Maven coordinate of the source artifact for which you want to generate Javadoc.
-sr URL | Maven repository containing sources JAR. Defaults to http://repo1.maven.org/maven2/
-mr URL | Maven repositories containing Code Recommenders models. Defaults to http://download.eclipse.org/recommenders/models/luna/
-p PROVIDER_ID [PROVIDER_ARGUMENTS] | Choose Livedoc Providers and configure filter thresholds for recommendations, each Provider respectively.
-mc GROUP_ID:ARTIFACT_ID:VERSION | Optional Maven coordinate for the model artifact used to enhance the Javadoc. If not set, the source coordinate is used for the model.
-u URL | Maven repository to upload Javadoc JAR to
-d DIRECTORY | Destination directory for the Javadoc HTML output files
-debugging | Debug output messages, includes full stack traces
-encoding ENCODING | Overwrite auto-guessed encoding of source files
-highlight | Shall Livedoc highlight its output with a yellow background?
-jar | Want to have a zipped JAR output for Javadoc? Warning: Requires -d <DIRECTORY>
-noDependencies | Disable download of additional classpath entries via direct maven dependencies
-splitIndex | Splits the index file into multiple files, alphabetically, one file per letter, plus a file for any index entries that start with non-alphabetical symbols.
-subpackages "org.eclipse org.apache …" | A list of packages to Javadoc (full recursively), otherwise all found packages in the maven coordinate will be used.
-transitive | Shall Livedoc try to also resolve transitive dependencies instead of only direct ones?
-verbose | Output messages about what Livedoc is doing
-Xdoclint (all|none|[-]<group>) | Enable or disable specific checks for problems in javadoc comments, where <group> is one of the following: accessibility, html, missing, reference, or syntax.
-Xresume | Fast mode, no unpacking, no temp directory cleaning WARNING: For testing purposes only.

### Providers

Livedoc comes with five Providers (more coming soon!), which enhance the HTML output with several code statistics:

Name | Id | Description
--- | --- | ---
Method Overrides | ovrm | Methods frequently overridden by subclasses
Method Override Patterns | ovrp | Groups of methods which are frequently overriden together
"Call Also" | call-also | Subsequent method calls
Method "Self Calls" | selfm | Methods called frequently by implementors of a method (from subclasses)
Class "Self Calls" | selfc | Methods called frequently by subclasses
Google Analytics Provider | analytics | Adds Google Analytics to your Javadoc web page

You can decide which Providers you want to use with the `-p` flag, e.g. with

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p ovrm -p selfm

only the Providers Method Overrides and Method "Self Calls" are being used. If you want to simply use all available Providers (dependent on the plug-ins loaded), you can also skip the flag

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102

and all Providers will be used, except for the Google Analytics Provider, which is a special case: This Provider will add the necessary Javascript Code for the Google Analytics API to your web page, including your own Tracking-ID. (see also: "Provider Configuration")

The following code statistic Providers are currently available:

#### Method Overrides

The Method Overrides Provider shows recommendations for methods which are frequently overridden by subclasses of a Class:

#### Method Override Patterns

The Method Override Patterns Provider shows recommendations for groups of methods which are frequently overridden together by subclasses of a Class:

#### "Call Also"

The "Call Also" Provider shows recommendations of the form "People who called Foo also called Bar":

#### Method "Self Calls"

Method "Self Calls" are calls to methods which are made frequently by implementors of a super method:

#### Class "Self Calls"

Class "Self Calls" are methods called frequently by subclasses:

### Provider Configuration

Regardless of whether you’re using all providers or just one, you may want to filter their output. Otherwise you may end with hundreds of recommendations for a class, but with extremly low percentage values. Additionally, some Providers may need additional arguments to fit your special needs. For this, each Provider can be configured on its own, using command-line arguments.

#### Let’s start with filtering the output:

Livedoc provides three thresholds for this, configurable for each provider:

Threshold | Flag | Description | ovrm | ovrp | call-also | selfm | selfc
--- | --- | --- | --- | --- | --- | --- | ---
Number | -top | Show only the top N recommendations | X | X | X | X | X
Times observed | -times | Show only recommendations with N or more times observed |  | X |  | X | X
Percentage | -percent | Show only recommendations with N or more percent | X |  | X | X | X

However, not every threshold can be used with every Provider, only the ones with "X" marked in the table above. Using a threshold with an unsupported Provider won’t have any consequences to filtering.

To specify a threshold for a Provider, you’ll have to pass it as a Provider option, this happens with square brackets, following the syntax `-p PROVIDER_ID [-FLAG VALUE]`, e.g.

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p ovrm [-percent 15] -p ovrp [-top 5]

Now, only method override recommendations with 10% or more will be shown, plus the top 5 method override patterns.

#### Additional Provider Arguments:

The three thresholds for filtering are optional, Providers can also work without these filter values. However, there are Providers which `need` additional arguments to work, thus they won’t be loaded if these arguments aren’t present.

An example (and currently the only one) for this is the Google Analytics Provider. With this Provider, you can let Livedoc easily insert the Tracking-Code for Google Analytics into your Javadoc HTML code.

To achieve this, you will have to provide your Google Analytics ID as an Argument:

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -p analytics [ANALYTICS_TRACKING_ID]

If you select the Google Analytics Provider (with "-p analytics"), but don’t provide an ID, the following error will be printed:

    15:22:47.865 [main] ERROR o.e.r.i.l.javadoc.RecommendersTaglet - Provider "AnalyticsProvider" couldn’t be set up, skipping it.
        Message: org.kohsuke.args4j.CmdLineException: Argument "WEB_PROPERTY_ID" is required
        Exception: org.eclipse.recommenders.livedoc.providers.LiveDocProviderException

and the Provider won’t be loaded!

### DocLint feature support

Livedoc supports the new DocLint feature of JDK 8 Javadoc. However, a slight syntax modification to the original Javadoc -Xdoclint is needed to configure the option in Livedoc:

    ./livedoc ... [-Xdoclint (all|none|[-]<group>) ...]

E.g., with

    ./livedoc ... -Xdoclint all -Xdoclint -accessibility

you would enable all groups of checks except the accessibility checks. For a reference of the group checks, see the JDK 8 Javadoc Documentation: http://docs.oracle.com/javase/8/docs/technotes/tools/unix/javadoc.html#BEJEFABE
NOTE: The DocLint feature in Livedoc is an opt-in instead of an opt-out like in original JDK 8 Javadoc. Meaning by default, the -Xdoclint option is disabled and no type of group checks will be made. See also this blog post by Stephen Colebourne why we made this decision.

### Direct and transitive dependency resolution

Let’s take a look at this Livedoc command line:

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -sr http://download.codetrails.com/livedoc/sources/ -mr http://download.eclipse.org/recommenders/models/luna-m3/ -d ./output -highlight

Livedoc can handle the `org.eclipse.jface` package, however it has one problem: During the Livedoc run, you’ll see command line outputs like the following:

    /tmp/livedoc/sources/org.eclipse.jface-3.8.102-sources/org/eclipse/jface/window/ApplicationWindow.java:37:
    error: package org.eclipse.swt.widgets does not exist
    import org.eclipse.swt.widgets.Control;
                                  ^
    …
    …
    4396 warnings
    Done.

This means that the artifact `org.eclipse.jface:org.eclipse.jface:3.8.102` has dependencies on the package `org.eclipse.swt.widgets`. Without these dependencies, Javadoc won’t be able to use fully qualified names for these Classes and thus, Livedoc can’t include Code Recommenders statistics for them.

Livedoc tries to resolve (download) these dependencies from your source code repository using the source artifact’s meta information. However, Livedoc will only try to resolve `direct` dependencies. Transitive dependencies won’t be resolved by default, as this could take some time (and bandwidth). You can force Livedoc to resolve transitive dependencies using the `-transitive` flag:

    ./livedoc org.eclipse.jface:org.eclipse.jface:3.8.102 -sr http://download.codetrails.com/livedoc/sources/ -mr http://download.eclipse.org/recommenders/models/luna-m3/ -d ./output -highlight -transitive

The result will be less warnings , fully qualified names in the HTML output and more statistics from Code Recommenders! Of course, Livedoc can only use dependencies which are included in your source code repository.