PacktPub is celebrating 10 years – get any eBook for $10

My favorite IT publisher Packt Publishing has prepared awesome sale! They are celebrating 10 years of providing great learning and information materials for IT professionals. They are offering all of their ebooks and videos at just $10 each, also you can buy as many copies as you like until July 5. Act quickly as there are only 2 days left until this promotion ends!

You can find more information under this link:

http://bit.ly/1vw0B9t

Alfresco and high CPU usage

I finally found a solution for the high CPU usage bug in Alfresco. I still don’t know why it happens but at least I know how to deal with it so it does not make the whole environment unusable.

So the problem can be described as high CPU usage peaks which happen every 15 seconds. Every 15 seconds (default value) SOLR is asking Alfresco if it should index new data,and even though there is nothing to index you can observe that at least one CPU core usage jumps to 100% for 2-4 seconds.

Solution is to recreate SOLR indexes and clear the cached models, you can do this like this:


rm -fr alfresco/alf_data/solr/archive
rm -fr alfresco/alf_data/solr/workspace
rm alfresco/alf_data/solr/archive-SpacesStore/alfrescoModels/*
rm alfresco/alf_data/solr/workspace-SpacesStore/alfrescoModels/*

(backup them just in case!)

Then restart the server and give it some time to reindex data. Depending on the repository size it can take even very long hours.

Then the issue should be gone!

Bizarre error with DA 6.5

For some reason Documentum Administrator 6.5 SP3 started failing with following error:

12:13:18,400 DEBUG [main] com.documentum.debug.Trace - An error occurred during application intialization. Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
12:13:18,402 ERROR [main] com.documentum.web.common.Trace - An error occurred during application intialization : Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
java.lang.IllegalStateException: Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
  at com.documentum.services.config.impl.PrimaryElementsByScopeDictionary.put(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseConfigFile(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.loadConfig(Unknown Source)
  at com.documentum.operations.impl.config.ConfigManager.initializeConfigService(ConfigManager.java:438)
  at com.documentum.operations.impl.config.ConfigManager.newConfigService(ConfigManager.java:335)
  at com.documentum.operations.impl.config.ConfigManager.getConfigService(ConfigManager.java:499)
  at com.documentum.operations.impl.config.ConfigManager.lookupPopulator(ConfigManager.java:223)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.populate(OperationNodeTreeBuilder.java:574)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.add(OperationNodeTreeBuilder.java:64)
  at com.documentum.operations.DfOperation.add(DfOperation.java:320)

It took me some time to find the culprit and decided describe it because I couldn’t find any solution on the Internet.

So this error will show up only when you load a dump from another repository, in such cases some duplicate objects will be created and this will cause the problem.

The solution is to remove duplicate objects delete.xml and delete_dm_message_archive.xml from /System/Operations/1/Config/Message and /System/Operations/1/Config/Attachment folders.

And to avoid such issues in future it is recommended to rename /System/Operations folder before loading the exported content.

Documentum, Docbrokers and NAT

This note is for my future reference. Each time I am setting up docbroker translation rulesΒ  I am confused how it should be done correctly. This note will help me to do it quickly in future πŸ™‚ Perhaps this will be also useful for others…

So, first of all, if you want to have a docbase behind NAT you need two docbrokers. The first one is an internal one which serves all requests from internal network (behind the firewall/NAT). The second docbroker serves all external requests, coming from, for example, VirtualBox/VMWare host clients or other computers on the external network.

It means that the external docbroker will need to have some extra translation configuration.

Configuration for the external docbroker will be:

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1491

[TRANSLATION]
port=<CS service port>=<CS service port>
host=<external IP>=10.0.2.15

where the <external IP> is the IP of the host on which the Virtualbox/VMWare is running (or the public IP). Port number is the port on which the docbase is listening (CS service port, you can find it out by checking /etc/services on Linux hosts).

The internal docbroker has standard configuration, the only difference is that it should be (obviously) running on a different port than the external docbroker.

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1489

Extremely important bit: for each docbroker there must be TWO ports allocated. For example, if you have an internal docbroker on port 1489, then the external docbroker CAN NOT be configured on 1490, the next free port will be 1491. If you don’t do that you will see an error like:


Documentum Docbroker: dispatcher failed (104) on line 1070 -- non-fatal

This is described in http://support.emc.com/kb/110976

It is also important to remember about configuration of the projection targets in server.ini. Both docbrokers must be defined like this:


[DOCBROKER_PROJECTION_TARGET]
host = server
port = 1489
[DOCBROKER_PROJECTION_TARGET_1]
host = server
port = 1491

When configuring docbase on VirtualBox VM some additional NAT configuration has to be done. Usually VirtualBox has two interfaces, localhost and the NAT one (ethX, with IP for example 10.0.2.15). It is important to configure Content Server and docbrokers to listen on the NAT interface. To configure it in Content Server edit server.ini and add “host=10.0.2.15”.

The last piece of configuration is the port forwarding:

After setting it up restart the system and hopefully it should be working all fine πŸ™‚

 

New Stamper website.

A few days ago I have enabled the new Stamper website: http://stamper.metasys.pl. A lot has changed but the most importantly Stamper demo is available live. You can actually see it for yourself!

There are two interesting places you can visit. First one is the Stamper Configurator, a new, separate from Alfresco, web application where you can configure Stamper. This application is using CMIS to access the repository (because all configuration objects are stored there). I have plans to support Nuxeo soon, some preliminary work has been done and it is looking promising! The Stamper Configurator is available here. Login credentials are admin/admin.

Second place is the Alfresco repository itself. You can log in there as admin/admin (by the way the repository is reinitialized every day so no damage can be done there! :-). There are a few PDF documents there as well as two simple Stamper configurations, the first one contains cover page and page layer and the second one just the page layer. To stamp a PDF file just use the Stamp document option in a document’s context menu.

I will be also attending the Alfresco DevCon 2011 in London in November, I hope that this will be a good place to chat about Alfresco, Stamper and other interesting topics. Fell free to ask questions!

 

cmis-in-batch & Documentum 6.7

I have just pushed changeset which enables Documentum 6.7 support in cmis-in-batch. Generally speaking OpenCMIS supports Documentum 6.7 out of the box, but cmis-in-batch required some additional modifications due to the way how Documentum handles paths to objects. Documentum allows multiple objects with the same name in a single folder and therefore it is impossible to uniquely address an object using a typical path. To solve this problem Documentum uses different path segments, each object name is represented in following form:

with name: 0900000b80001234_docname
without name: 0900000b80001234_

Other limitation of Documentum 6.7 CMIS is that it exposes only CMIS object types (document, folder and relationship) and not custom ones, therefore the fail-when-type function will not work as expected.

You can get the latest source code using Mercurial like this:


hg clone https://cmis-in-batch.googlecode.com/hg/ cmis-in-batch 

cmis-in-batch

I have just uploaded an initial version of tool which tries to simplify data import and manipulation in CMIS enabled repositories.

More details as well as source code (under Apache License 2.0) are available here:
https://code.google.com/p/cmis-in-batch/

It is still Work In Progress but it works, at least now, with Alfresco. I have just finished installation of Documentum 6.7 with newly added CMIS support and I am going to test it as soon as possible.

Groovy is great!

It is not only great mix of useful and convenient language constructs of Python, Ruby and other languages but it also turned out that it is a very good code obfuscator πŸ™‚ Here is an example, code below:


    public static IDfSession getSession(String user,
        String password, String docbase) throws DfException {

        IDfClientX clientx = new DfClientX();
        IDfClient client = clientx.getLocalClient();
        IDfLoginInfo li = new DfLoginInfo();
        li.setUser(user);
        li.setPassword(password);
        return client.newSession(docbase, li);
    }

was translated to:


    public static IDfSession getSession(String user, String password, String docbase)
        throws DfException
    {
        CallSite acallsite[] = $getCallSiteArray();
        com.documentum.com.IDfClientX clientx = ((com.documentum.com.IDfClientX) (acallsite[0].callConstructor($get$$class$com$documentum$com$DfClientX())));
        IDfClient client = (IDfClient)ScriptBytecodeAdapter.castToType(acallsite[1].call(clientx), $get$$class$com$documentum$fc$client$IDfClient());
        com.documentum.fc.common.IDfLoginInfo li = ((com.documentum.fc.common.IDfLoginInfo) (acallsite[2].callConstructor($get$$class$com$documentum$fc$common$DfLoginInfo())));
        acallsite[3].call(li, user);
        acallsite[4].call(li, password);
        return (IDfSession)ScriptBytecodeAdapter.castToType(acallsite[5].call(client, docbase, li), $get$$class$com$documentum$fc$client$IDfSession());
    }

Awesome πŸ™‚

“Fix Groovy Imports” in NetBeans

I was very frustrated with the fact that the Groovy & Grails plugin for NetBeans does not have the “Fix Imports” functionality: the option in the menu is simply grayed out. I started looking into the issue (and the NetBeans code…) and it turned out that the feature is actually implemented, it is just incorrectly connected to the menu option.

In order to fix it you have to define a new shortcut which will call the”Fix Groovy Imports” function. Just navigate to Tools/Options/Keymap and then change the shortcut as shown below:

That’s all πŸ™‚

Data importer for CMIS repositories

When developing Stamper I missed a tool which would allow me to easily import some files into a repository. Alfresco provides import/export functionality through ACP files which can be used for that purpose but what about other repositories? What about Nuxeo, Documentum? Documentum has Composer/Application Builder, Nuxeo perhaps also has it’s own mechanisms, but I wanted one tool which would work with all those repositories without need to create a separate installation package for each system.

Hopefully there is a CMIS standard which can make this happen.

CMISetuper (working name :-)) will be a tool which will be able to:

  • connect to any repository which supports CMIS. This will be possible thanks to the OpenCMIS library.
  • import files
  • create folders
  • validate presence of types, objects, folders
  • modify content (replace, version etc.)
  • link and unlink objects from folders

Those actions will be described by a declarative language named SDL – Simple Declarative Language. Below is a fragment of a script:


execute {
    import-files "/myFiles" {
        "image1.jpg" cm_name="Some title" metasys_taken_on_date=2009/10/05
        "image2.jpg" metasys_color=false metasys_taken_on_date=2011/01/05
    }

    import-file "/Documents" "someotherfile.doc"
    delete-file "/Old document.doc" all-versions="true"

    replace-content "/Pictures/picture.jpg" "/tmp/newContent.jpg"
    update-properties "/Documents/someotherfile.doc" cm_title="another title"
    link-to-folder "/myHouse.jpg" "/myFiles"
    unlink-from-folder "myHouse.jpg" "/myFiles"
}

Basically the tool will execute scripts in a few stages:

  • pre-validate stage – this stage can check presence of types, files, folders etc. in repository and fail when they are present (or not).
  • prepare stage – at this stage some preparations can take place, for example creation of folders for which objects will be imported.
  • execute stage – this is the main stage which takes care of actual data importing, linking etc.
  • validation stage – the last stage which checks whether everything was imported correctly

Some global settings will be available which will control overwrite mode as well as whether to stop on errors.
I have also plan to implement rollback mode but perhaps not in the first version πŸ™‚

At this moment I would say that around 60% of above functionality is implemented and I hope to have rest ready by end of this week, so stay tuned for updates!

If you have some new ideas please share them with me.