www.archive-org-2013.com » ORG » D » DACAPOBENCH

Choose link from "Titles, links and description words view":

Or switch to "Titles and links view".

    Archived pages: 19 . Archive date: 2013-03.

  • Title: DaCapo Benchmarks Home Page
    Descriptive info: .. [Frameless version]..

    Original link path: /
    Open archive

  • Title: The DaCapo Benchmark Suite
    Original link path: /title.html
    (No additional info available in detailed archive for this subpage)

  • Title:
    Descriptive info: Main.. News.. TODO.. Benchmarks.. Benchmarks.. Download.. Usage.. Threading.. Soot.. DayTrader.. Problems.. Internals.. Mailing lists.. Analysis.. Paper.. CACM Paper.. Tech Report.. Talk.. 2006-10-MR2 Statistics.. Performance.. Mercurial Head.. 9.. 12 Bach Release.. 2006-10-MR2 Release..

    Original link path: /menu.html
    Open archive
  •  

  • Title: The DaCapo Benchmark Suite
    Descriptive info: Overview.. This benchmark suite is intended as a tool for Java benchmarking by the programming language, memory management and computer architecture communities.. It consists of a set of.. open source.. ,.. real world.. applications with non-trivial memory loads.. The initial release of the suite was the culmination of over five years work at eight institutions, as part of the.. DaCapo research project.. , which was funded by a National Science Foundation ITR Grant, CCR-0085792.. Three years of development have gone into the 2009 release.. This work has been funded by the ANU, the Australian Research Council and a generous donation from Intel.. Our suite evolves to maintain its relevance.. It is therefore.. essential that you.. cite the version number.. associated with the release.. in any use of the benchmark, and as a courtesy to the developers, we ask that you.. please cite the paper.. from OOPSLA 2006 describing the suite:.. Blackburn, S.. M.. , Garner, R.. , Hoffman, C.. , Khan, A.. , McKinley, K.. S.. , Bentzur, R.. , Diwan, A.. , Feinberg, D.. , Frampton, D.. , Guyer, S.. Z.. , Hirzel, M.. , Hosking, A.. , Jump, M.. , Lee, H.. , Moss, J.. E.. B.. , Phansalkar, A.. , Stefanovic, D.. , VanDrunen, T.. , von Dincklage, D.. , and Wiedermann, B.. The DaCapo Benchmarks: Java Benchmarking Development and  ...   head.. Dec 24, 2009.. After three years of development, the new release of the DaCapo benchmark suite is finally available.. You can grab it.. here.. Please be sure to read the.. release notes.. before using this release.. This release includes new workloads, deletes many old ones, overhauls all other workloads.. The release also includes many improvements to the harness and commandline interface.. See.. news.. for a list of past news.. Plans.. Since the release of the 9.. 12-bach release, we have been working toward the next major release of the suite, currently scheduled to appear in.. the first half of 2012.. We plan to make beta releases in the weeks prior to the major release.. We are continually looking at improving and possibly removing existing workloads.. mailing list.. if you have suggestions for improving the existing benchmarks.. Feedback.. We invite feedback on the suite from the community.. Please use the researchers'.. License.. Each of the constituent benchmarks is open source or public domain.. The original source for each of the benchmarks is automatically downloaded by building our.. source distribution.. The copyright information for each benchmark is available via the.. -i.. option to the test harness.. The suite itself is available under the.. Apache license.. For questions or comments please use the.. researchers mailing list.. Copyright 2001-2009 by the.. DaCapo Project.. ,.. All Rights Reserved..

    Original link path: /main.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: 2012-02.. 2009-12.. 2009-07.. Our continuous performance regression pages (.. release.. and.. development.. ) have been updated to include time stamps and direct links to the jars used.. This means you can always download the current development.. jar.. 2009-06.. After a huge development effort, we now have early drafts of two benchmarks based on the Apache DayTrader J2EE workload:.. tradebeans.. tradesoap.. At this stage both benchmarks are unstable on some platforms, including our testing environment.. We strongly encourage feedback (please use the mailing list).. Please try them out by downloading the development.. Read more about these workloads.. As part of a major push to get a beta version of the suite ready, we have started culling those benchmarks that will not appear in the next release.. The following benchmarks have been removed:.. antlr.. (jython now uses antlr internally),.. bloat.. (we have a number of program analysis tools),.. chart.. (batik has some similarity as a vector graphics renderer),.. hsqldb.. (replaced by derby).. A new benchmark,.. avrora.. , has been added to the suite.. Avrora was proposed by Ben Titzer from Sun.. Avrora.. is a parallel discrete event simulator that performs cycle accurate simulation of a sensor network.. Avrora exhibits interesting patterns of parallelism, and is unlike any of the existing DaCapo workloads, so makes a very interesting addition.. Please try it out by downloading the development jar.. Thank you very much Ben!.. Almost all benchmarks have been updated to reflect the latest publicly available releases.. For jython, lusearch, luindex, and pmd this marks significant changes.. Only eclipse and xalan remain to be updated.. 2008-11.. In anticipation of the upcoming release, we have created a.. TODO list.. of work that needs to be done before the upcoming release.. Please send email to the mailing list or directly to Steve Blackburn if you think you can help.. 2008-08.. A new paper describing the development of the DaCapo benchmark suite and broader issues of methodology.. appears.. in August 2008 CACM.. 2008-07.. We perform comprehensive, 12-hourly performance regressions, running various VMs against the DaCapo.. svn head.. (for the upcoming release), and against the.. 2006 release.. 2008-06.. Slides from a presentation discussing the development of the DaCapo suite and the broader topic of evaluation methodology are now.. available.. Cliff Click pointed us to the.. fragger.. widgit, which artificially injects fragmentation into a heap.. We are considering including this as a runtime option in the DaCapo suite, as it could be very for those using DaCapo for memory management and locality work.. Thanks Cliff.. 2008-01.. Stability and performance comparisons among a number of JVMs running against the DaCapo suite can now be found.. , with tests against the current DaCapo development head  ...   next 6-12 months for possible inclusion in the next release of the suite.. Note that SunFlow requires Java 1.. 5 or later.. 2006-12-26.. A minor maintenance release (dacapo-2006-10-MR1) is now.. This includes a bug fix and minor enhancements to the interface.. 2006-10-25.. The first full release of the dacapo benchmark suite (dacapo-2006-10) is now available.. This release includes the following bug fix:.. Fixed a bug in pmd sources.. Three of the pmd input sources included a non-UTF-8 character where the resulting behavior is undefined.. Some JVMs (correctly) produced different behavior for these files.. The offending character has been removed from the three input files.. 2006-10-15.. We have produced release candidates in anticipation of our initial full release around October 23.. These are now available for.. download.. and evaluation.. Feedback recieved prior to October 20 may be included in the full release.. New features for release candidates include:.. Further improvements to the validation mechanism.. We now accommodate the DOS "\" path separator--hopefully this is the last of the OS-specific issues.. We have added a new build target.. split-deps.. This allows those who build from source to separate out dependencies into a separate jar (this feature was added after feedback on the mailing list, and is used to enable whole program analysis).. [Thanks to Eric Bodden of McGill!].. Improvements to the harness.. Fixed a bug that prevented output from subsequent benchmarks being displayed when executing more than one benchmark at a time.. Improved usage message.. [Thanks to Vladimir Strigun of Intel].. The.. eclipse.. benchmark now uses a self-contaied dummy jre.. for its build tasks, so it should work on any jvm without special action (previously a third-party jre needed to be specified on the command line of jvms which Eclipse did not recognise).. Minor improvements to the build scripts.. This includes updates to source URLs, refactoring, comments.. 2006-10-07.. We have made a new beta release, beta-2006-10, which has a number of improvements including:.. xalan.. benchmark has been re-written.. This version has a more realistic workload.. We also force the use of version 2.. 4.. 1 and ensure that xalan is used (rather than a bundled xslt processor).. [Thanks to Kev Jones of Intel!].. The validation mechanism has been completely overhauled.. The validation mechanism ensures that benchmarks only "PASS" if they complete correctly.. The harness should now work properly on any OS, and deals with the users' pwd which creeps into some benchmark output.. [Thanks to Robin Garner of ANU!].. lusearch.. benchmark has been improved.. , so that each query thread includes a mix of unique and shared queries (previously all queries were unique/non-shared).. The build system has been improved.. considerably and is a little better documented..

    Original link path: /news.html
    Open archive

  • Title: DaCapo Benchmarks TODO List
    Descriptive info: Major TODO Items For Next Release.. The DaCapo benchmark suite is a community project, developed by the research community, for the research community.. The quality of the workloads depends on community critque and community contributions.. There is a large amount of work to be done.. Please feel free to help by contributing to one or more of the following tasks (use the mailing list or email Steve Blackburn directly to co-ordinate your efforts).. Revise This List For Next Release.. DayTrader Benchmarks.. Stability.. The workload was originally developed for DaCapo under MacOS X using the default JVM on that platform, Sun's 1.. 5.. 0 HotSpot VM.. Once completed we started testing on a other platforms, with other production JVMs and made two significant observations:.. The workloads are stable on most VMs using H2 database, but were noted as unstable with Derby.. Performance.. Under Mac OS X (using Derby), CPU utilization is near 100% (top shows "180%" or so on a dual-core machine), however under ubuntu, running similar HotSpot JVM on similar hardware, CPU utilization drops precipitiously.. However, shifting to H2 the performance seems stable so this is no longer a priority.. Feedback On Existing Benchmarks.. Feedback on the existing benchmarks will help us determine which benchmarks to drop from the upcoming release (the release will include new benchmarks and drop some of the existing benchmarks).. If you have any comments, please let us know.. Current candidates for exclusion from the upcoming suite include.. (has some notable idiosyncrasies, and is not extensively deployed),.. (superseded by.. ), and possibly.. and/or.. Incorporate New Benchmarks.. The new suite includes a number of new benchmarks.. Done Jun 2009.. Batik.. Done Dec 2007.. Sunflow.. Done Jan 2007.. H2.. Done.. H2 driven with the Derby TPC-C like workloads.. Done Dec 2009.. Wiki.. Sergey Salishev of Intel has created a workload that simulates a cloud computing environment, including a wiki..  ...   to ensure it is at the most recent stable release for that workload.. Xalan.. We have used a very particular version of Xalan (2.. 1), on advice from Kev Jones of Intel (who provided us with our current version of the workload).. Kev's rationale may have become dated with newer releases of Xalan.. We should revisit this decision for the next release.. Eclipse.. In addition to updating the Eclipse version, it would be great to investigate, and if necessary address a problem identified by Matt Arnold in June 2007:.. I have some info about the Dacapo Eclipse benchmark that you may be interested in.. If I should contact someone else instead, feel free to let me know.. If you run eclipse for a large number of iterations, the performance forms a saw tooth, degrading significantly (a factor of 10 or more), then jumps back to normal.. An excel graph of the performance over time is attached below.. (Dacapo version 2006-10.. jar).. It happens on both Sun's and IBM VM's.. During the slow iterations the program is spending most of its time in jitted code.. The problem is the following two methods:.. org/eclipse/jdt/internal/compiler/util/WeakHashSetOfCharArray.. add([C)[C.. org/eclipse/jdt/internal/compiler/util/WeakHashSet.. add (Ljava/lang/Object;)Ljava/lang/Object;.. Both of these methods have a linear search through some kind of linked list of weak references.. Here's my guess at what is happening: This list grows over time and the linear searches eventually becomes a huge bottleneck.. Eventually some memory threshold is crossed and the VM clears the weak references, and performance goes back to normal.. This is clearly crappy code (linear searching) but it could also be a benchmark bug.. Is it possible that this data structure should have been re-initialized between iterations, and the iterating nature of the driver is creating a problem is unlikely to exist in the real application?.. Copyright 2001-2008 by the.. Last modified: Sun Jun 21 13:46:36 EST 2009..

    Original link path: /todo.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: The benchmarks.. The DaCapo-9.. 12-bach benchmark suite, released in 2009, consists of the following benchmarks:.. simulates a number of programs run on a grid of AVR microcontrollers.. produces a number of Scalable Vector Graphics (SVG) images based on the unit tests in Apache Batik.. executes some of the (non-gui) jdt performance tests for the Eclipse IDE.. fop.. takes an XSL-FO file, parses it and formats it, generating a PDF file.. h2.. executes a JDBCbench-like in-memory benchmark, executing a number of transactions against a model of a banking application, replacing the hsqldb benchmark.. jython.. inteprets a the pybench Python benchmark.. luindex.. Uses lucene to indexes a set of documents; the works of Shakespeare and the King James Bible.. Uses lucene to do a text search of keywords over a corpus of data comprising the works of Shakespeare and the King James Bible.. pmd.. analyzes a set of Java classes for a range of source code problems.. sunflow.. renders a set of images using ray tracing..  ...   documents into HTML.. We have a default size of all benchmarks, and where applicable a small and large size.. We highly recommend using small or default for testing, and reporting default any performance analysis.. The DaCapo 2006 benchmark suite, initial release in 2006, consists of the following benchmarks:.. parses one or more grammar files and generates a parser and lexical analyzer for each.. performs a number of optimizations and analysis on Java bytecode files.. uses JFreeChart to plot a number of complex line graphs and renders them as PDF.. executes some of the (non-gui) jdt performance tests for the Eclipse IDE.. takes an XSL-FO file, parses it and formats it, generating a PDF file.. executes a JDBCbench-like in-memory benchmark, executing a number of transactions against a model of a banking application.. We have provided 3 inputs (small, default, large).. We highly recommend using small for testing, and either reporting default or large in any performance analysis.. For some benchmarks default and large workloads are identical..

    Original link path: /benchmarks.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: To use the suite, all you need is the dacapo jar, which can be downloaded.. (167MiB).. We have a source distribution available.. (10.. 7 MiB).. To build the dacapo jar from these sources you will need the apache 'ant' tool - see the supplied README file for details.. This contains the DaCapo benchmark software but none of the third-party software.. The ant task 'sources' downloads the required packages..

    Original link path: /download.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: Our suite will evolve to maintain its relevance.. essential that the version number associated with the release be cited in any use of the benchmark,.. and we ask that you please cite the 2006 OOPSLA paper describing the suite:.. Running benchmarks.. Execute the jar without any arguments to get usage information:.. java -jar dacapo-9.. 12-bach.. jar.. To run one of the benchmarks, use the command followed by the benchmark name, for example avrora:.. jar avrora.. Guidance.. When reporting results, be sure to report the precise version of the suite, as well as any command line arguments used.. The benchmark harness allows control over the number of iterations executed (allowing observation of the benchmark warm-up).. The harness also allows the number of client (external) threads to be controlled for many benchmarks (some benchmarks are intentionally single-threaded).. Note that we sharply differentiate.. internal.. external.. concurrency.. The former (internal) is an intrinsic property of the benchmark and is not user-controlable.. For example the avrora benchmark uses threads to represent  ...   is partitioned.. The tradebeans and tradesoap benchmarks exhibit both forms of concurrency, with their server components using thread pools to manage requests (internal concurrency, not user controlable), and while the harness specifies the number of clients which concurrently make requests to the server (external concurrency, user controlable).. For all benchmarks that exhibit external concurrency, the level of concurrency is by default set at the number of available processors.. This may be overriden at the command-line, either with an explicit number of threads or with a user-specified multiplier against the number of available processors.. It is important that you explicitly report the number of threads used when you override the harness defaults.. Callbacks.. The harness allows for pluggable callbacks which are executed before and after benchmark execution.. These allow (for example) JVM-specific statistics collection to be started and stopped.. Here.. is an example benchmark callback class.. The distributed jar file contains one pre-built callback,.. MMTkCallback.. , which brackets the timing run with calls to MMTk's benchmark harness..

    Original link path: /usage.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: 12-bach benchmark suite, released in 2009, has a range of multithreaded benchmarks.. The following table characterizes the nature of the threading in the existing benchmarks.. Driven by a single external thread, but it is internally multithreaded with each simulated element using a thread (i.. e.. each node in a grid of simulated nodes is threaded).. Avrora demonstrates a high volume of fine granularity interactions between simulator threads.. Driven by a single external thread and the bulk of its internal work is single threaded.. It does use worker threads for transcoding and rendering images and the use of Java2D also greats a worker thread.. Driven by a single external thread it is internally multithreaded.. However, some worker thread activity seems to be serialised, while others seem to engage in some fine granularity interactions.. As such, eclipse exhibits periods of little concurrency and brief periods of moderate granularity concurrency (further investigation is required).. Multithreaded, it is driven by one client thread per hardware thread and internally has a server thread for each client thread as well as other support threads.. The number of  ...   client thread per hardware thread, requiring little interaction between threads.. Driven by a single client thread it is internally multithreaded using one worker thread per hardware thread.. Multithreaded, driven by a client thread per hardware thread, each thread processing an available tile of work at a time and another thread created due to the use of Java2D.. Multithreaded, driven by a client thread per hardware thread, each client thread performing a series of requests which are dealt with by a server thread for that client thread.. Multithreaded, driven by a client thread per hardware thread, each client thread performs requests from an effective queue of requests.. Those requests are dealt with by corresponding server threads.. The pool of server threads created by the geronimo server is large, most threads are idle.. Same as for tradebeans.. Multithreaded, explicitly driven by the number of hardware threads available, each thread taking an element from a work queue.. The DaCapo benchmark suite, initially released in 2006, has a handful of multithreaded benchmarks.. 20 threads during the later (query) part of the workload.. 32 threads.. 8 threads..

    Original link path: /threads.html
    Open archive

  • Title: The DaCapo Benchmark Suite
    Descriptive info: DaCapo 9.. 12 "bach".. The "bach" release of DaCapo uses a non-trivial class-loading mechanism and reflection to bootstrap each benchmark for which the harness is invoked.. Static analysis of DaCapo therefore requires static-analysis tools to (1) obtain access to all classes that are loaded at runtime, and (2) obtain information about all reflective calls.. Eric Bodden now.. provides the TamiFlex tool.. that allows you to gather both class files and a reflection log file.. Soot 2.. 0.. and later can use the reflection log file to fully statically analyze the DaCapo "bach" release.. this Technical Report.. for details.. This page explains in detail.. how to analyze DaCapo "bach" with Soot.. Deprecated: DaCapo 2006-10.. The remainder of this page describes how to use the 'xdeps' packaging of the old (out of date) 2006-10 release of the DaCapo benchmark suite (eg.. dacapo-2006-10-MR2-xdeps.. zip.. ) to process them with.. The information may be of interest to anyone performing.. ahead-of-time whole program analyses.. Please note.. that we strongly recommend that you use the regular jar packaging of the suite (eg.. dacapo-2006-10-MR2.. jar.. ) unless you specifically need to use the 'xdeps' repackaging as described here.. Special thanks to Eric Bodden of McGill for helping develop support for Soot and contributing the following.. Processing the DaCapo benchmarks with Soot.. The DaCapo  ...   are contained in the source distribution as well.. Those stub classes adhere to the name pattern.. dacapo.. bm.. Main.. Application classes and library classes.. Soot users are probably aware that Soot distinguishes between application classes and library classes (only the former are optimized, see the.. Soot tutorial.. for details).. In order to support this model, there is an "xdeps" packaging of DaCapo available, where each benchmark,.. bm.. , is compiled separately into.. two.. jars.. One file has the name.. and holds all application classes for the benchmark.. The other file has the name.. benchmark -deps.. and holds all library classes for the benchmark.. This packaging is avaliable as a binary download (.. dacapo- RELEASE -xdeps.. ), or via an ant target for a source build.. Applying Soot to the DaCapo benchmarks.. In order to apply Soot to a benchmark.. BM.. you basically need the following command line (or you can use the following.. script.. ):.. java -Xmx512m -cp.. /abc-complete.. jar \.. soot.. Main \.. -process-dir ${TEMP} \.. -soot-class-path ${JRE}/lib/rt.. jar:${JRE}/lib/jce.. jar:${JRE}/lib/jsse.. jar:${B}-deps.. jar:${TEMP} \.. -main-class dacapo.. ${BM}.. ${CUSTOM_OPTIONS}.. Here,.. TEMP.. points to some temporary directory,.. JRE.. to the Java runtime library you want to analyze and.. to the name of one of the DaCapo benchmarks.. In case of any questions, feel free to email.. Eric Bodden..

    Original link path: /soot.html
    Open archive



  •  


    Archived pages: 19