Overview
The Advanced Java Module for extends the Basic Java Module. The advanced module discovers dependencies between packages, classes, methods, and fields. It adds functionality for deep analysis of Java sources such as navigating method call graphs, inheritance hierarchies, and usage graphs.
The module can be used with Hammurapi or other tools to perform deep analysis of Java sources.
The module supports analysis of all versions of Java sources. It requires Java 5 to run.
Features
- Discovery and navigation of inheritance hierarchies.
- Discovery and navigation of method call graphs.
- Discovery and navigation of dependencies between classes, methods, and fields.
- Aggregation of dependencies at package level. Navigation of dependencies between packages.
How it works
At the load time the module analyzes java classes bytecode and stores information about dependencies into the database. It stores the following informations:
- Inheritance relationship between classes.
- References from classes to other classes, methods and fields.
- Method call relationships between methods.
At the analysis time, when ClassDefinition, InterfaceDefinition, or other language elements which reference other elements, are instantiated, the module instantiates extended implementations of these classes, e.g. ClassDefinitionEx, InterfaceDefinitionEx. The extended implementations contain methods to navigate dependencies between language elements.
Installation
Install Mesopotamia as described in First steps.
Download mesopotamia-java-advanced-2.3.1.zip, unzip and copy mesopotamia-advanced-java.jar and bcel-5.2.jar from the Advanced Module lib folder to the Mesopotamia installation modules\Java\lib folder.
Then create tables and load Java Advanced module definitions using the following command
java -cp lib\hgcommons.jar;lib\hsqldb.jar;lib\mesopotamia.jar;modules\Java\lib\mesopotamia-java.jar;modules\Java\lib\mesopotamia-java-advanced.jar biz.hammurapi.mesopotamia.lang.javax.util.InitDatabase [<host>:<port>/<alias>] [<user>] [<password>]
You can optionally specify database host, port, alias, user name, and password. By default, the utility loads to the HSQLDB server running on localhost on default port with default user. See HSQLDB documentation for more details.
Loading
In order to perform "advanced" loading of source files, we need to pass classloader to the createScan() method as environment entry as show in the code snipped below in bold.
package org.mesopotamia.firststeps;
import java.io.File;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;
import javax.sql.DataSource;
import org.mesopotamia.FileSourceIterator;
import org.mesopotamia.Language;
import org.mesopotamia.LanguageSelector;
import org.mesopotamia.Repository;
import org.mesopotamia.RepositoryFactory;
import org.mesopotamia.Scan;
import org.mesopotamia.Source;
import org.mesopotamia.SourceIterator;
import biz.hammurapi.mesopotamia.lang.javax.Constants;
import biz.hammurapi.sql.SQLProcessor;
import biz.hammurapi.sql.hypersonic.HypersonicIdentityRetriever;
import biz.hammurapi.sql.hypersonic.HypersonicServerDataSource;
public class Load {
/**
* @param args
*/
public static void main(String[] args) throws Exception {
DataSource ds = new HypersonicServerDataSource("localhost", "mesopotamia", "", null);
SQLProcessor processor=new SQLProcessor(ds, null);
biz.hammurapi.persistence.ChunkingStringStorage stringStorage = new biz.hammurapi.persistence.ChunkingStringStorage();
stringStorage.setDatasource(ds);
stringStorage.setIdentityManager(new HypersonicIdentityRetriever());
stringStorage.setChunkSize(Integer.MAX_VALUE);
stringStorage.start();
RepositoryFactory factory=new RepositoryFactory(processor, null, null, null, stringStorage);
Repository repository=factory.createRepository("Mesopotamia sources");
System.out.println("Repository: "+repository.getId());
File file=new File("C:\\Tools\\Mesopotamia\\src");
Collection rootFiles=new ArrayList();
rootFiles.add(file);
SourceIterator si = new FileSourceIterator(rootFiles);
final Language targetLanguage = new Language("Java", "5", "Java 5");
LanguageSelector ls = new LanguageSelector() {
public Language select(Source source) {
return source.getName().endsWith(".java") ? targetLanguage : null;
}
};
long start = System.currentTimeMillis();
Map environment = new HashMap();
environment.put(Constants.CLASS_LOADER_KEY, Load.class.getClassLoader());
Scan scan=repository.createScan(si, ls, "As of "+new Date(), "SHA", environment, null);
System.out.println("Done with scan " +scan.getId() +" in "+ ((System.currentTimeMillis()-start)/1000)+" seconds.");
stringStorage.stop();
}
}
There are two levels of advanced loading:
- Inheritance and references between classes in a form of field types, method return and parameter types. In other words, information which can be obtained through reflection API.
- All references between classes and method call graph. This information is obtained by interrogating class file bytecode.
By default, classes from java and sun packages and their sub-packages (JDK classes) are loaded at level one. All other class are loaded at level two. This behavior is controlled by two environment properties:
- Constants.EXCLUDE_REFERENCES_PACKAGES_KEY (value "java.lang.ClassLoader:ENVIRONMENT:EXCLUDE_REFERENCES_PACKAGES"). This entry shall contain either a string collection with packages to be excluded from level 2 loading, or a string which contains a comma-separated list of packages to be excluded. Exclusion applies to package and its sub-packages. By default, if this value is not specified, "java" and "sun" packages and their sub-packages are excluded.
- Constants.INCLUDE_REFERENCES_PACKAGES_KEY (value "java.lang.ClassLoader:ENVIRONMENT:INCLUDE_REFERENCES_PACKAGES"). This entry shall contain either a string collection with packages to be included into level 2 loading, or a string which contains a comma-separated list of packages to be included. If this property is not specified then all packages which are not excluded are included.
The sample below specifies that only package biz.hammurapi and its sub-packages shall be loaded to the level 2.
environment.put(Constants.INCLUDE_REFERENCES_PACKAGES_KEY, "biz.hammurapi");
Analysis
NOTE: The advanced module uses soft references.
Set -Xms as well as -Xmx to improve performance.
During analysis advanced information shall be obtained either from the Scan object, or from advanced language elements. The "root" class of the advanced API is ScanEx. From instance of this class all other advanced info can be obtained. ScanEx is obtained from Scan instance as shown in the code snippet below in bold.
package biz.hammurapi.test;
import java.util.Timer;
import javax.sql.DataSource;
import org.mesopotamia.RepositoryFactory;
import org.mesopotamia.Scan;
import biz.hammurapi.logging.ConsoleLogger;
import biz.hammurapi.logging.Logger;
import biz.hammurapi.mesopotamia.lang.javax.Dependency;
import biz.hammurapi.mesopotamia.lang.javax.DependencyVisitor;
import biz.hammurapi.mesopotamia.lang.javax.Info;
import biz.hammurapi.mesopotamia.lang.javax.PackageInfo;
import biz.hammurapi.mesopotamia.lang.javax.ScanEx;
import biz.hammurapi.mesopotamia.lang.javax.TypeInfo;
import biz.hammurapi.sql.SQLProcessor;
import biz.hammurapi.sql.hypersonic.HypersonicIdentityRetriever;
import biz.hammurapi.sql.hypersonic.HypersonicServerDataSource;
import biz.hammurapi.util.ThreadPool;
public class VisitScan {
public static void main(String[] args) throws Exception {
Logger logger=new ConsoleLogger(ConsoleLogger.DEBUG);
// Create empty data source.
DataSource ds = new HypersonicServerDataSource("localhost:1720/mesopotamia", "sa", "", null);
SQLProcessor processor=new SQLProcessor(ds, null);
Timer timer = new Timer();
biz.hammurapi.persistence.ChunkingStringStorage stringStorage = new biz.hammurapi.persistence.ChunkingStringStorage();
stringStorage.setDatasource(ds);
stringStorage.setIdentityManager(new HypersonicIdentityRetriever());
stringStorage.setTimer(timer);
stringStorage.setChunkSize(Integer.MAX_VALUE);
stringStorage.start();
final ThreadPool threadPool = new ThreadPool(5, Thread.NORM_PRIORITY, 1000, null, "Loading pool");
threadPool.start();
// Create repository factory
RepositoryFactory factory=new RepositoryFactory(processor, threadPool, logger, timer, stringStorage);
Scan scan = factory.getScan(processor.projectSingleInt("SELECT MAX(ID) FROM MESOPOTAMIA.SCAN", null));
ScanEx scanEx = ScanEx.getScanEx(scan);
TypeInfo typeInfo = scanEx.findType("biz.hammurapi.config.ComponentBase");
TypeInfo[] st = typeInfo.getSupertypes();
for (int i=0; i<st.length; ++i) {
System.out.println(st[i].getFullyQualifiedName());
}
TypeInfo mc = scanEx.findType("biz.hammurapi.metrics.MeasurementConsumer");
System.out.println(mc.isAssignableFrom(typeInfo));
System.out.println(mc.getTypeId());
System.out.println("Clients");
typeInfo.getPackage().visitClients(
new DependencyVisitor<PackageInfo>() {
public boolean visit(PackageInfo target, int distance, double strength) {
System.out.println(target+" "+distance+" "+strength);
return true;
}
});
for (Dependency<PackageInfo> d: mc.getPackage().getClients()) {
System.out.println(d.getStrength()+" "+d.getTarget().getFullyQualifiedName());
}
System.out.println("Suppliers");
for (Dependency<PackageInfo> d: mc.getPackage().getSuppliers()) {
System.out.println(d.getStrength()+" "+d.getTarget().getFullyQualifiedName());
}
typeInfo.getPackage().visitSuppliers(
new DependencyVisitor<PackageInfo>() {
public boolean visit(PackageInfo target, int distance, double strength) {
System.out.println(target+" "+distance+" "+strength);
return true;
}
});
typeInfo.visitSuppliers(new DependencyVisitor<Info>() {
public boolean visit(Info target, int distance, double strength) {
System.out.println(target.getFullyQualifiedName()+" "+distance+" "+strength);
return true;
}
});
threadPool.stop();
timer.cancel();
System.out.println("Done with " +scan.getId());
}
}
Use in Hammurapi
Inspectors
The easiest way to write inspectors which check advanced features is to have them inspect advanced implementations, e.g. ClassDefinitionEx instead of ClassDefinition.
To perform scan-wide analysis, e.g. to build package dependency matrix, write inspect(Scan) method, and obtain ScanEx from Scan: ScanEx scanEx = ScanEx.getScanEx(scan);
Loading to advanced levels
From release 5.4.0 Hammurapi task <load> element supports classPath attribute, classPath nested element and environmentEntry nested elements. If classpath is set and the database is upgraded for advanced loading, then <load> element will perform advanced loading. Packages to include/exclude into/from level 2 advanced loading can be specified through environmentEntry nested elements.
Limitations
This release doesn't implement all the planned funcitionality. In particular:
- Anonymous and local classes are not linked to source representations.
- Field, method, and method call graph information is stored in the database, but not yet exposed through the analysis API.
- Advanced type information is linked to type and package definitions, but not references.
- findMethod() and findField() methods are not implemented.
- MethodInfo and FieldInfo interfaces are not finalized.
Licensing
The free version of the module cannot be used for any commercial or corporate purpose. To purchase a commercial license contact