Bizarre error with DA 6.5

For some reason Documentum Administrator 6.5 SP3 started failing with following error:

12:13:18,400 DEBUG [main] com.documentum.debug.Trace - An error occurred during application intialization. Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
12:13:18,402 ERROR [main] com.documentum.web.common.Trace - An error occurred during application intialization : Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
java.lang.IllegalStateException: Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
  at com.documentum.services.config.impl.PrimaryElementsByScopeDictionary.put(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseConfigFile(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.loadConfig(Unknown Source)
  at com.documentum.operations.impl.config.ConfigManager.initializeConfigService(ConfigManager.java:438)
  at com.documentum.operations.impl.config.ConfigManager.newConfigService(ConfigManager.java:335)
  at com.documentum.operations.impl.config.ConfigManager.getConfigService(ConfigManager.java:499)
  at com.documentum.operations.impl.config.ConfigManager.lookupPopulator(ConfigManager.java:223)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.populate(OperationNodeTreeBuilder.java:574)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.add(OperationNodeTreeBuilder.java:64)
  at com.documentum.operations.DfOperation.add(DfOperation.java:320)

It took me some time to find the culprit and decided describe it because I couldn’t find any solution on the Internet.

So this error will show up only when you load a dump from another repository, in such cases some duplicate objects will be created and this will cause the problem.

The solution is to remove duplicate objects delete.xml and delete_dm_message_archive.xml from /System/Operations/1/Config/Message and /System/Operations/1/Config/Attachment folders.

And to avoid such issues in future it is recommended to rename /System/Operations folder before loading the exported content.

Documentum, Docbrokers and NAT

This note is for my future reference. Each time I am setting up docbroker translation rules  I am confused how it should be done correctly. This note will help me to do it quickly in future 🙂 Perhaps this will be also useful for others…

So, first of all, if you want to have a docbase behind NAT you need two docbrokers. The first one is an internal one which serves all requests from internal network (behind the firewall/NAT). The second docbroker serves all external requests, coming from, for example, VirtualBox/VMWare host clients or other computers on the external network.

It means that the external docbroker will need to have some extra translation configuration.

Configuration for the external docbroker will be:

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1491

[TRANSLATION]
port=<CS service port>=<CS service port>
host=<external IP>=10.0.2.15

where the <external IP> is the IP of the host on which the Virtualbox/VMWare is running (or the public IP). Port number is the port on which the docbase is listening (CS service port, you can find it out by checking /etc/services on Linux hosts).

The internal docbroker has standard configuration, the only difference is that it should be (obviously) running on a different port than the external docbroker.

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1489

Extremely important bit: for each docbroker there must be TWO ports allocated. For example, if you have an internal docbroker on port 1489, then the external docbroker CAN NOT be configured on 1490, the next free port will be 1491. If you don’t do that you will see an error like:


Documentum Docbroker: dispatcher failed (104) on line 1070 -- non-fatal

This is described in http://support.emc.com/kb/110976

It is also important to remember about configuration of the projection targets in server.ini. Both docbrokers must be defined like this:


[DOCBROKER_PROJECTION_TARGET]
host = server
port = 1489
[DOCBROKER_PROJECTION_TARGET_1]
host = server
port = 1491

When configuring docbase on VirtualBox VM some additional NAT configuration has to be done. Usually VirtualBox has two interfaces, localhost and the NAT one (ethX, with IP for example 10.0.2.15). It is important to configure Content Server and docbrokers to listen on the NAT interface. To configure it in Content Server edit server.ini and add “host=10.0.2.15”.

The last piece of configuration is the port forwarding:

After setting it up restart the system and hopefully it should be working all fine 🙂