cmis-in-batch released with data generation mode

I have just pushed proper 1.0 release of the cmis-in-batch tool to github and bintray:

https://github.com/karolbe/cmis-in-batch

https://bintray.com/karolbe/metasys/cmis-in-batch

Here is a quote from the README.md:

Data generation is a useful feature that allows bulk importing of test data documents into CMIS compatible repository. Additionally it can populate metadata of documents with values coming from predefined dictionaries.

Sample script for generating thousands of documents can look like this one below.

Here is a brief description of what the script does:

* it will load three dictionaries from files /tmp/disciplines, /tmp/types, /tmp/subtypes. The dictionaries are simple text files where values are separated by new line characters. From the dictionary values Cartesian product will be calculated so for example, having three dictionaries:

1. level1A, level1B
2. level2A, level2B
3. level3A, level3B
following combinations will be generated:


[level1A, level2A, level3A]
[level1A, level2A, level3B]
[level1A, level2B, level3A]
[level1A, level2B, level3B]
[level1B, level2A, level3A]
[level1B, level2A, level3B]
[level1B, level2B, level3A]
[level1B, level2B, level3B]

* it will import each file in content-path location (“/media/kbryd/Media/work/sample_data/department”) to a location in repository defined with linking-rule: /Repository/${discipline}/static/${doctype}/sub/${docsubtype} – each ${} variable will be replaced by a value coming from appropriate dictionary.
* naming-rule defines what the object name should be. It can use variables from the dictionaries plus a few additional: ${file_name}, ${file_size}, ${file_path}, ${file_ext}, ${file_mime}
* mapping defines the mapping for populating metadata of each document, e.g. in this case discipline attribute will be populated with value of discipline.


   generate-random-data "set1" {
   doc-type "cara_document"
   linking-rule "/Repository/${discipline}/static/${doctype}/sub/${docsubtype}"
   naming-rule "${file_name} - ${doctype}"
   content-path "/media/kbryd/Media/work/sample_data/department"

   mapping {
       discipline {
          "${discipline}"
       }
       doc_type {
          "${doctype}"
       }
       doc_subtype {
          "${docsubtype}"
       }
   }

   dictionaries {
      discipline "/tmp/disciplines"
      doctype "/tmp/types"
      docsubtype "/tmp/subtypes"
   }
}

And that’s all! Have fun using it! 🙂

Bizarre Composer error

Composer is good when it works, but when it stops is a real PITA to understand what is going on. Sometimes cleaning a project helps, sometimes, re-importing does the magic, but sometimes nothing works. Almost.

So I had this issue that a perfectly fine Composer project one day simply stopped building. At all. All I was getting was this error message:

'Dar Install Options Set' of 'Dar Def-Methods' must be set default.dardef /Methods/dar org.eclipse.emf.ecore Problem

Turned out that project settings (upgrade/overwrite etc.) *somehow* vanished from default.dardef file, so I have added it back:

    <darInstallOptionsSet>
      <location xsi:type="installparameter:FolderParameter" href="urn:com.emc.ide.folderparameter/Dar+Install+Folder?artifactURI=file:/C:/Composer/Methods/Artifacts/Installation%20Parameters/darinstallfolder.parameter#//@dataModel"/>
      <upgradeOption xsi:type="dardef:UpgradeOption"/>
    </darInstallOptionsSet>

And it started working fine again…

Repoint 0.2.0 released

I have just released a new repoint release, version 0.2.0. Head to the repoint’s github project to get the executables for Linux and Windows. If you would like package for Mac OS please leave a comment and I will add it as well.

https://github.com/karolbe/repoint-r/releases/tag/v0.2.0

What’s new? Well, in the 0.1.0 version I have added a new dialog that shows ACL in a nice and readable way:Selection_059The feature is available from the right click menu from the tree and also from the DQL results viewer. In future this feature will also allow editing of ACLs.

In 0.2.0 I have added one more feature that I had on my TODO list since forever

Selection_058

It is now possible to add a docbroker and a docbase from that docbroker without need to edit dfc.properties. You can still use dfc.properties, but if need to quickly connect to a different docbase which is not in your dfc.properties then now it is possible 🙂

If you have ideas, bug reports etc. please use github or comments below to let me know

 

D2 Lockbox & VirtualBox

D2 Lockbox can be a real PITA to install but it surely is even more PITA when it is installed in a VM. Thing is that even if you have a properly installed D2 Lockbox it will stop functioning after each VM reboot.

It took me long hours to notice what is wrong, I was changing the setup, double checking it with the documentation a few times and everything seemed just fine…Funny enough after changing the setup I was doing a reboot – “just in case” 🙂

So, the lesson I have learnt is that  VirtualBox VM should not be restarted after Lockbox was configured. It is better to use “VM Pause” otherwise D2.lockbox file will have to be re-generated.

 

 

Documentum D2: How to display a dialog from a D2-Plugin?

Sometimes it is useful to display some message to the user from the code running on the server side (in D2-Plugin). At the first sight there doesn’t seem to be a way to do this but thanks to Dariusz R. (thanks!) who has decompiled half of D2 code 🙂 we have learned that it is actually possible. And it is also extremely simple:

The trick is to use D2fsExceptionManager.throwD2SilentException(D2fsContext context, String eventName, String messageToPublish) exception with appropriate event name and message, for example:


D2fsExceptionManager.throwD2SilentException(context, "D2_ACTION_DISPLAY_DIALOG", "DIALOG_NAME==VeryImportantDialog!!CHANNEL_EVENT==D2_ACTION_DISPLAY_DIALOG");

That’s all! Try it for yourself.

Customizing Documentum D2

I have been working on customization of D2 for the last 1.5 years and I think that I am in a good position to share some knowledge I managed to gather. I am going to start a series of blog posts explaining some less obvious tricks that I find useful.

So, lets start with customizing D2 menus…

There are two ways of doing it, one way involves using D2-Config, you can simply click on “Go to…/D2 Menu” and then configure order of menu items, change their conditions etc.
This is fine, but this is not always the best way of doing it. For example, what if you would like to add more advanced conditions which are not available in vanilla D2? Or what if you would like to dynamically create menu items? This simply can’t be done by using just the D2-Config. In more advanced projects it is often better to have the menus customized via XML in a D2 plugin than in D2-Config, this also allows better control of changes (as all files are in VCS some sort).

Okay, enough introduction, lets see some example. Lets assume that we would like to add a static D2 menu that will open some dialog.
In order to achieve this goal it is good to understand how D2 plugins work. At the moment lets only focus on a typical plugin folder structure:

blog-d2-plugin-tree

The menu layout XML file is stored in src/main/resources/xml/menu, so lets create a file MenuContextDelta.xml there with following content:


<?xml version="1.0" encoding="utf-8"?>
<delta>
    <insert position-after="menuContextEdit">
        <menuitem id="menuContextSampleStatic">
            <dynamic-action
                    class="com.emc.d2fs.dctm.ui.dynamicactions.actions.ShowDialog"
                    dialog="SampleDialog"/>
        </menuitem>
    </insert>
</delta>

The name of this file is important, it is a ‘delta’ file containing differences that merged with the base MenuContext.xml (that is stored in D2FS4DCTM-WEB-4.5.jar) will produce the final MenuContext.xml.

What the XML code does is pretty simple, we define a new menu item, that will be inserted after menuContextEdit menu item (that is ‘Edit’) in the Context menu (this is implied by name of the MenuContextDelta.xml file). When clicked, the new menu item will show dialog named ‘SampleDialog’.

Now, what if we would like to have a dynamic menu item? Lets say the requirement is to list available custom actions e.g “Publish to system A”, “Publish to system B” etc. when the menu is selected.

In order to implement it we need two files: first one adds the main menu item that contains reference to the second file (please note src=”PublishSubmenu”), second one points to the Java class that will generate available submenu options based on the current document selection (as available publishing systems depend on the selected document):

File: MenuContextDelta.xml


<?xml version="1.0" encoding="utf-8"?>
<delta>
    <insert position-after="menuContextEdit">
        <menuitem id="menuContextSampleSubmenu" src="PublishSubmenu">
        </menuitem>
    </insert>
</delta>

File: PublishSubmenu.xml


<?xml version="1.0" encoding="utf-8"?>
<delta>
    <dynamic-menuitem id="submenuPublish" class="com.metasys.d2.menu.PublishMenu">
    </dynamic-menuitem>
</delta>

And finally the Java class that will create submenu items:


package com.metasys.d2.menu;

import java.util.HashMap;
import java.util.List;
import java.util.Map;

import com.documentum.fc.client.IDfSysObject;
import com.emc.common.java.xml.XmlNode;
import com.emc.d2fs.dctm.ui.dynamicactions.actions.ShowDialog;
import com.emc.d2fs.dctm.web.services.D2fsContext;
import com.emc.d2fs.utils.AttributeUtils;

/**
 * Created by kbryd on 11/01/2015.
 */
public class PublishMenu extends SimpleDynamicMenu {

  public XmlNode getXmlMenuItem(D2fsContext context) throws Exception {
    XmlNode result = new XmlNode();
    IDfSysObject sysObject = (IDfSysObject)context.getFirstObject();
    AvailablePublishingSystemsModel model = PublishingUtils.getSystems(context.getSession(), sysObject);
    List<String> configNames = model.getSystemsAsList();

    for(int i = 0; i < configNames.size(); i++) {
      String configName = configNames.get(i);
      String label = configName;
      XmlNode node = result.appendChildNode("menuitem");
      node.setAttribute("id", "genid_" + i);
      node.setAttribute("label", label);

      Map<String, Object> attributes = new HashMap();
      attributes.put("DIALOG_NAME", "ConfirmPublishDialog");
      attributes.put("DIALOG_LIST_PARAM",
          "target_system" + AttributeUtils.SEPARATOR_VALUE + "label" + AttributeUtils.SEPARATOR_VALUE
              + "oam_id");
      attributes.put("target_system", configName);
      node.setAttribute("action", new ShowDialog().getAction(context, attributes));
    }
    return result;
  }
}

There is one small interesting thing going on there…I pass the target_system name to the dialog so it can be displayed as part of the confirmation message:


      Map<String, Object> attributes = new HashMap();
      attributes.put("DIALOG_NAME", "ConfirmPublishDialog");
      attributes.put("DIALOG_LIST_PARAM",
          "target_system" + AttributeUtils.SEPARATOR_VALUE + "label" + AttributeUtils.SEPARATOR_VALUE
              + "oam_id");
      attributes.put("target_system", configName);
      node.setAttribute("action", new ShowDialog().getAction(context, attributes));

Merely adding the value to the attributes Map is not enough to pass the value to the dialog. It is also important to add that attribute name to the DIALOG_LIST_PARAM value. Then in the dialog buildDialog method you can access the parameter value like this:


  @Override
  public XmlNode buildDialog(D2fsContext context, List<Attribute> attributes) throws Exception {
    XmlNode configDialog = XmlUtil.loadFromURL(context.getXmlDialogFile()).getRootXmlNode();

    DialogProcessor dialogProcessor = new DialogProcessor(context, configDialog);
    dialogProcessor.setLabelsBundle(context.getDialogBundle());
    Map<String, String> defaultValues = new HashMap<>();
    defaultValues.put("target_system", context.getParameterParser().getStringParameter("target_system"));
    dialogProcessor.setDefaultValues(defaultValues);

    XmlNode dialogNode = dialogProcessor.getDialog();
    return dialogNode;
  }

Nice thing about default values in dialogs is that the values can be automatically used in *.properties files.

Lets have a look at a ConfirmPublishDialog layout XML:


<?xml version="1.0" encoding="utf-8"?>
<dialog id="ConfirmPublishDialog" width="410" height="200" resizable="true" buttons_right="false">
    <content>
        <comment id="confirmMessage" html_content="true" condition_visible="true"/>
    </content>
    <buttons>
        <button id="buttonOk" type="submit" action="validDialog()" />
        <button id="buttonCancel" type="reset" action="cancelDialog()" />
    </buttons>
</dialog>

The comment element has id ‘confirmMessage’. This id also points to the resource in ConfirmPublishDialog_en.properties in src/main/resources/strings/dialog/ConfirmPublishDialog.

And the promised trick is that you can use $value constructs there like this:


ConfirmPublishDialog=Publication
buttonOk=Publish
buttonCancel=Cancel
confirmMessage=Please confirm that you want to publish selected document to $value(target_system)

This saves some additional effort with passing the value down to the view.

I hope this was useful!

Documentum Repoint Resurrected!

I guess everyone using Documentum on a daily basis knows what dqMan and repoint applications are. Life of a Documentum developer would be very difficult if those tools were not available. I tried so many times to get used to dqMan but I simply couldn’t, there is just something wrong with the User Interface…So even though repoint application felt like ‘beta’ version I was still preferring using repoint to dqMan.

I’ve had this plan since long time ago to restart development of repoint as it seems like the original author abandoned the project. And it finally happened. I have just committed a Maven friendly repoint project to github:

https://github.com/karolbe/repoint-r

If you would like to play with it you can grab the sources and use Maven to build it, just run:

mvn clean install

and after some time you will find repoint application for your architecture (linux, windows, macos) in following folder: repoint-eclipse-repository/target/products/Repoint

So far I have only removed some deprecated RCP code but I have some bigger plans for that application. Depending on my spare time I am going to work on:

* ACL editor
* Trusted Content Services support

and of course fix issues and improve usability.

Bizarre error with DA 6.5

For some reason Documentum Administrator 6.5 SP3 started failing with following error:

12:13:18,400 DEBUG [main] com.documentum.debug.Trace - An error occurred during application intialization. Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
12:13:18,402 ERROR [main] com.documentum.web.common.Trace - An error occurred during application intialization : Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
java.lang.IllegalStateException: Lookup dictionary 'scope-global': Duplicate element 'step[id=delete_object_processing]', scope='aspect='*', type='dm_message_container', format='*', domain='*', application='operations'', config files='C:\Documentum\local\process3237028314044277291.tmp\0\060007c580002e80.xml' and 'C:\Documentum\local\process3237028314044277291.tmp\0\060007c5800002b9.xml'
  at com.documentum.services.config.impl.PrimaryElementsByScopeDictionary.put(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseConfigFile(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.initialiseApp(Unknown Source)
  at com.documentum.services.config.impl.ConfigService.loadConfig(Unknown Source)
  at com.documentum.operations.impl.config.ConfigManager.initializeConfigService(ConfigManager.java:438)
  at com.documentum.operations.impl.config.ConfigManager.newConfigService(ConfigManager.java:335)
  at com.documentum.operations.impl.config.ConfigManager.getConfigService(ConfigManager.java:499)
  at com.documentum.operations.impl.config.ConfigManager.lookupPopulator(ConfigManager.java:223)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.populate(OperationNodeTreeBuilder.java:574)
  at com.documentum.operations.impl.OperationNodeTreeBuilder.add(OperationNodeTreeBuilder.java:64)
  at com.documentum.operations.DfOperation.add(DfOperation.java:320)

It took me some time to find the culprit and decided describe it because I couldn’t find any solution on the Internet.

So this error will show up only when you load a dump from another repository, in such cases some duplicate objects will be created and this will cause the problem.

The solution is to remove duplicate objects delete.xml and delete_dm_message_archive.xml from /System/Operations/1/Config/Message and /System/Operations/1/Config/Attachment folders.

And to avoid such issues in future it is recommended to rename /System/Operations folder before loading the exported content.

Documentum, Docbrokers and NAT

This note is for my future reference. Each time I am setting up docbroker translation rules  I am confused how it should be done correctly. This note will help me to do it quickly in future 🙂 Perhaps this will be also useful for others…

So, first of all, if you want to have a docbase behind NAT you need two docbrokers. The first one is an internal one which serves all requests from internal network (behind the firewall/NAT). The second docbroker serves all external requests, coming from, for example, VirtualBox/VMWare host clients or other computers on the external network.

It means that the external docbroker will need to have some extra translation configuration.

Configuration for the external docbroker will be:

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1491

[TRANSLATION]
port=<CS service port>=<CS service port>
host=<external IP>=10.0.2.15

where the <external IP> is the IP of the host on which the Virtualbox/VMWare is running (or the public IP). Port number is the port on which the docbase is listening (CS service port, you can find it out by checking /etc/services on Linux hosts).

The internal docbroker has standard configuration, the only difference is that it should be (obviously) running on a different port than the external docbroker.

[DOCBROKER_CONFIGURATION]
host=10.0.2.15
port=1489

Extremely important bit: for each docbroker there must be TWO ports allocated. For example, if you have an internal docbroker on port 1489, then the external docbroker CAN NOT be configured on 1490, the next free port will be 1491. If you don’t do that you will see an error like:


Documentum Docbroker: dispatcher failed (104) on line 1070 -- non-fatal

This is described in http://support.emc.com/kb/110976

It is also important to remember about configuration of the projection targets in server.ini. Both docbrokers must be defined like this:


[DOCBROKER_PROJECTION_TARGET]
host = server
port = 1489
[DOCBROKER_PROJECTION_TARGET_1]
host = server
port = 1491

When configuring docbase on VirtualBox VM some additional NAT configuration has to be done. Usually VirtualBox has two interfaces, localhost and the NAT one (ethX, with IP for example 10.0.2.15). It is important to configure Content Server and docbrokers to listen on the NAT interface. To configure it in Content Server edit server.ini and add “host=10.0.2.15”.

The last piece of configuration is the port forwarding:

After setting it up restart the system and hopefully it should be working all fine 🙂