Archive

Archive for Grudzień 2009

Internationalization of Web DynPro components part 2 : resource bundles and PDF’s

2009-12-30 Komentarze wyłączone

In addition to S2X-based i18n you can use classic java properties-based i18n. In some cases it can be more convenient, f.g. property bundles are created by tools such as eclipse string externationalization.

But you must remember that string externationalization tool was designed for heavy client desktop application – in web systems ResourceBundle.getBundle and similar methods are unadequate – they return server locale which is in most cases unadequate, and not the locale the user is using!

The following code will use the user’s current locale:

public static String getString(String key) {
  // Get the locale of the current session
  Locale sessionLocale = WDResourceHandler.getCurrentSessionLocale();
  IWDResourceHandler resourceHandler = WDResourceHandler
    .createResourceHandler(sessionLocale);
  resourceHandler.loadResourceBundle
    BUNDLE_NAME, 
    WnioskiListMessages.class.getClassLoader() );
  try {
    return resourceHandler.getString(key);
  catch (MissingResourceException e) {
    return '!' + key + '!';
  }
}

You can use this code as replacement for Eclipse-generated code.

Another case to be dealt with is the usage of the localized strings. When they are used to display messages, it’s no problem, because SAP engine is using unicode encoding. However, with PDF it’s not so easy.

I will describe what to do if you use popular iText library. First if you add some text, you provide font. Fonts can be created via BaseFont.createFont. There are a couple of fonts ready to use by PDF engine, which names are defined in BaseFont class. However, when using this fonts, you can’t use IDENTITY-H encoding, which covers unicode character range. And when providing multilanguage version, you can’t expect that user will provide characters from limited range. Event with english version it’s the case. Imagine someone writes info about planned business trip to Russia and wants to provide the name of target city in Russian characters… The best solution would be to analyze each character and dynamically change font used in PDF to those covering the given part of text. Good enough in most cases would be to use font that has good coverage of unicode characters. For asian scripts there would be a great problem to find such font, but I will consider european languages only for my needs.

When you digg into iText tutorials, you’ll find that one that uses BaseFont.createFont using as second character windows’ standard arial unicode font. This font covers very large number of national character sets. The problem is, the target machine for your application need not to be Windows… and you don’t have rights to embed that font in PDF or your application.

But there is a plenty of free fonts available. The font I’ve found good enough is FreeSerif from freefonts package, which can be downloaded from http://savannah.gnu.org/projects/freefont/.

After downloading I’ve made jar file from it, placing all fonts in package named fonts. Then I’ve added it to library component, wrapped it with J2EE Server Library DC and deployed to server. After that I was able to embed and use my fonts using following code:

String fontPath = getClass().getClassLoader()
  .getResource("fonts/FreeSans.ttf")
  .toExternalForm();
BaseFont baseFont = BaseFont.createFont(
  fontPath, BaseFont.IDENTITY_H, BaseFont.EMBEDDED)

Internationalization of Web DynPro component made easier

2009-12-28 Komentarze wyłączone

Internationalization (i18n) is very important issue for most of web systems, and almost every significant framework provides its tools for it. So this is with SAP’s Web DynPro. The tool is using modified XLIFF format (known as S2X), in which for each national version there exists single XLF file. Naming convention is similar to this from Java properties file. File of name MyView.wdview.xlf will have German version MyView.wdview_de.xlf, Russian version MyView.wdview_ru.xlf etc.

Editing XML files is no way so simple and human-friendly as editing properties files, so SAP provides its tool for that job (s2x editor). However, the tool is buggy and very unergonomic to use. Additionally the initial job to translate everything, including banal formulas such as month and city names, standard menu positions etc is boring, replicative and unchallenging.

This is why I’ve came up with idea of automating this process. I’ve used the most automatic option which means translating all values using online translating service, Google Translate f.g. However, exchanging GoogleTranslate class with other implementation, f.g. reading text files prepared by human translator, it can be used to make translation process more convenient efficient.

Here goes the code that automatically translates given resource to a few national versions.







package pl.linfo.lang.xlf;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.List;

import org.apache.xerces.parsers.DOMParser;
import org.w3c.dom.DOMImplementation;
import org.w3c.dom.Document;
import org.w3c.dom.Element;
import org.w3c.dom.Node;
import org.w3c.dom.NodeList;
import org.w3c.dom.bootstrap.DOMImplementationRegistry;
import org.w3c.dom.ls.DOMImplementationLS;
import org.w3c.dom.ls.LSOutput;
import org.w3c.dom.ls.LSSerializer;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;

import pl.linfo.lang.translate.GoogleTranslator;

public class SapXlfProcessor {
  
  private String sourceLang = "pl";
  private String destLang = "en";
  
  private Document doc;
  
  public void process(String xlfPaththrows SAXException, IOException, ClassCastException, ClassNotFoundException, InstantiationException, IllegalAccessException {
    File xlfFile = new File(xlfPath);
    DOMParser parser = new DOMParser();
    parser.parse(new InputSource(new FileInputStream(xlfFile)));
    doc = parser.getDocument();
    Node root = doc.getFirstChild();
    processNodes(root);
  }
  
  public void dump(PrintStream psthrows ClassCastException, ClassNotFoundException, InstantiationException, IllegalAccessException {
    DOMImplementation implementation= DOMImplementationRegistry.newInstance()
    .getDOMImplementation("XML 3.0");
    DOMImplementationLS feature = (DOMImplementationLSimplementation.getFeature("LS",
    "3.0");
    LSSerializer serializer = feature.createLSSerializer();
    LSOutput output = feature.createLSOutput();
    output.setByteStream(ps);
    serializer.write(doc, output);
  }

  private void processNode(Node item) {
    if ("body".equals(item.getNodeName())) {
      processBody(item);
    else if ("file".equals(item.getNodeName())) {
      ((Elementitem).setAttribute("source-language", destLang);
      processNodes(item);
    else if ("originalLocale".equals(item.getNodeName())) {
      ((Elementitem).setAttribute("xml:lang", destLang);
    else {// if ("header".equals(item.getNodeName()) || "giltDirectives".equals(item.getNodeName())) {
      processNodes(item);
    }
  }

  private void processNodes(Node item) {
    NodeList nl = item.getChildNodes();
    for (int i=0;i<nl.getLength();i++) {
      processNode(nl.item(i));
    }
  }

  private void processBody(Node item) {
    // process all groups
    List<Element> transUnits = new ArrayList<Element>();
    for (int i=0;i<item.getChildNodes().getLength();i++) {
      Node groupNode = item.getChildNodes().item(i);
      for (int j=0;j<groupNode.getChildNodes().getLength();j++) {
        Node unitNode = groupNode.getChildNodes().item(j);
        if ("trans-unit".equals(unitNode.getNodeName())) {
          transUnits.add((ElementunitNode);
        }
      }
    }
    // now we can do translation
    if (transUnits.size() == 0) {
      System.out.println("Nothing to translate");
      return ;
    }
    String[] values = new String[transUnits.size()];
    for (int i=0;i<transUnits.size();i++) {
      Element transUnit = transUnits.get(i);
      NodeList sources = transUnit.getElementsByTagName("source");
      if (sources.getLength() == 1) {
        values[i((Elementsources.item(0)).getTextContent();
      else 
        values[i""
    }
    String[] translated = new String[values.length];
//    List<String> toTranslate = new ArrayList<String>();
//    for (int i=0;i<values.length;i++) {
//      translated[i] = new GoogleTranslator().translate(text, sourceLang, destLang)
//    }
    try {
      translated = new GoogleTranslator().translate(values, sourceLang, destLang);
    catch (Exception e) {
      e.printStackTrace();
      return;
    }
    for (int i=0;i<transUnits.size();i++) {
      Element transUnit = transUnits.get(i);
      NodeList sources = transUnit.getElementsByTagName("source");
      if (sources.getLength() == 1) {
        ((Elementsources.item(0)).setTextContent(translated[i]);
      }
    }  
  }
  
  public static void main(String[] argsthrows Exception {
    String fileName = "MyView.wdview.xlf";
    String[] targetLangs = new String[]{"en""de""ru""uk"};
    SapXlfProcessor processor = new SapXlfProcessor();
    for (String targetLang : targetLangs) {
      processor.destLang = targetLang;
      processor.process(fileName);
      String dumpFile = new File(fileName).getName();
      dumpFile = "tmp/" + dumpFile.replace(".xlf""_"+targetLang+".xlf");
      processor.dump(new PrintStream(dumpFile));
    }
  }

}

Java2html


As you will notice, ukrainian language is referenced as uk, not ua. This can be quite confusing for users getting error Can’t set language. The full list of country codes used by SAP is listed in following PDF document.

The implementation of GoogleTranslate is using Google Translate API library and is listed here:






package pl.linfo.lang.translate;

import java.io.File;
import java.io.IOException;
import java.io.PrintStream;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Properties;

import net.htmlparser.jericho.Element;
import net.htmlparser.jericho.HTMLElementName;
import net.htmlparser.jericho.Source;
import net.htmlparser.jericho.TextExtractor;

import org.apache.commons.lang.StringEscapeUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.HTTP;

import com.google.api.translate.Language;
import com.google.api.translate.Translate;

public class GoogleTranslator {
  
  private HttpClient httpClient;
  
  private Source currentPage;
  
  public GoogleTranslator() throws IllegalStateException, IOException {
    httpClient = new DefaultHttpClient();
    httpClient.getParams().setParameter("http.protocol.expect-continue"false);
    httpClient.getParams().setParameter("http.connection.timeout"180*1000);

    Translate.setHttpReferrer("http://translate.google.pl&#34;);
    HttpGet httpget = new HttpGet("http://translate.google.pl&#34;);
    HttpResponse response = httpClient.execute(httpget);
    HttpEntity entity = response.getEntity();
    currentPage = new Source(entity.getContent());
  }
  
  public String[] translate(String[] text, String sourceLang, String destLangthrows Exception {
    
    Language sl = Language.fromString(sourceLang);
    Language tl = Language.fromString(destLang);
    
    // do grouping
    int ptr = 0;
    int len = 0;
    int max_len = 500;
    String[] result = new String[text.length];
    List<String> tmp = new ArrayList<String>();
    for (int i=0;i<text.length;i++) {
      
      int strlen = text[i].length();
      len += strlen;
      tmp.add(text[i]);
      
      if (i+1==text.length || len >= max_len) {
        System.out.println("Doing request with " + len + " bytes");
        String[] tmp2 = Translate.execute(tmp.toArray(new String[0]), sl, tl);
        for (int j=0;j<tmp2.length;j++)
          result[ptr+j= tmp2[j];
        ptr = i+1;
        len = 0;
        tmp.clear();
      }
    }
      return result;
  }   
  
}

Java2html