Cross Compiling Freeling


      Why?

      Well basically i like Freeling, i think is a quite good platform, and i like to use it in Windows. And actually i've seen only two ways to get a win32 version and i said myself, "What the hell!!" if everybody says that's complicated it must be very fun, and this is the main reason, just for fun. http://www.lsi.upc.edu/~nlp/freeling/index.php?option=com_simpleboard&Itemid=55&func=view&id=333&catid=5&limit=6&limitstart=0

      How?

      Well in order to get the binaries prepared for the win32 platform we have to follow those steps:
      • Installing MingW32
      • Downloading Pcre, Berkeley Database, Boost, Libcfg+, Omlet, Fries and Freeling
      • Cross-Compiling Everything
        • PCRE
        • BDB
        • Boost
        • LIbCFG+
        • Omlet
        • Fries
        • Freeling
      • Testing and Voila

      Installing MingW32

      First step is to install mingw and the binutils. In ubuntu to do so you should use this command: #sudo apt-get install mingw32 build-essential automake libtool subversion swig After that you can check the directory /usr/i586-mingw32msvc/ and the compiler version, if i do execute: #i586-mingw32msvc-c++ --version i586-mingw32msvc-c++ (GCC) 4.2.1-sjlj (mingw32-2) Copyright (C) 2007 Free Software Foundation, Inc.
      Now it seems we have the system ready. If you can't use deb, rpm or similar, visit http://www.mingw.org

      Downloading

      After the installation we can go with the downloading, to know exactly what we need to download you can follow the instructions at http://www.lsi.upc.edu/~nlp/freeling/index.php?option=com_wrapper&Itemid=63 And Following that list we need: LibCFG is included with Freeling. I personally used subversion as the origin of the sources from Freeling.

      Compiling

      Well let's do it. Actually i have a directory that looks like that: /boost_1_44_0 /boost-jam-3.1.18 /db-5.0.26.NC /freeling /fries /libcfg+-0.6.2 /omlet /pcre-8.10. And now will arrange everything to compile the sources and put the results (libraries) in the right place to make the final compilation (freeling). Note: If you prefer using mingw versions of the files you can avoid the compiling process. Yes, is truth, you can pick a binary distribution for windows, and place the files, usually lib and includes in the right places (/usr/i586-mingw32msvc/) and you will get the same result.

      How do we cross compile?

      To make a cross compilation we need to set those environment variables:
        • export CXX=i586-mingw32msvc-g++
        • export CC=i586-mingw32msvc-gcc
        • export AR=i586-mingw32msvc-ar
        • export RANLIB=i586-mingw32msvc-ranlib
        • export LD=i586-mingw32msvc-ld
        • export RANLIB=i586-mingw32msvc-ranlib
        With that and passing the variable --host=mingw32 to the configure tool everything should work.
      LibCFG+:
      ./configure --build=mingw --host=mingw32 ; make all ; sudo cp libcfg+.a /usr/i586-mingw32msvc/lib/ After this we will have the first library compiled. Easy, isn't.
      PCRE:
      ./configure --disable-shared --enable-utf8 --host=mingw32 --enable-unicode-properties CFLAGS="-mno-cygwin" && make sudo cp .libs/*.a /usr/i586-mingw32msvc/lib/ If you want can generate the dll too: make pcre.dll sudo cp pcre.dll /usr/i586-mingw32msvc/bin sudo cp pcre.h /usr/i586-mingw32msvc/include/ Note: If you find problems linking freeling maybe you need to set #define PCRE_STATIC 1 in config.h Note2: http://forums.badadev.com/viewtopic.php?f=4&t=724
      Berkeley Database:
      cd build_unix ../dist/configure --enable-mingw LIBCSO_LIBS=-lwsock32 LIBXSO_LIBS=-lwsock32 --disable-shared --enable-utf8 --enable-cxx && make sudo cp *.a /usr/i586-mingw32msvc/lib/ sudo cp ../build_windows/db_cxx.h /usr/i586-mingw32msvc/include/ sudo cp db.h /usr/i586-mingw32msvc/include/ If you are having problems with BdB maybe have to change to an older version as suggests Mathieu in this blog entry: http://mathieu.carbou.free.fr/wiki/index.php?title=Berkeley_DB_for_MinGW
      Boost:
      cd boost-jam-3.1.18 ./bootstrap.sh --without-icu cd boost_1_44_0 ./bjam toolset=gcc target-os=windows variant=release threading=multi threadapi=win32 link=shared runtime-link=shared --prefix=$PREFIX --user-config=user-config.jam -j 2--without-mpi --without-python -sNO_BZIP2=1 -sNO_ZLIB=1 --layout=tagged --build-type=complete stage Now to install you can choose or add install to the previous command or copy from stage the .a, and dll to the right places sudo cp stage/lib/*.a /usr/i586-mingw32msvc/lib sudo cp stage/lib/*.dll /usr/i586-mingw32msvc/bin sudo cp -R boost /usr/i586-mingw32msvc/include/
      Omlet:
      cd omlet aclocal; libtoolize; autoconf; automake -a ./configure --host=mingw32 && make sudo cp src/libomlet/.libs/libomlet.a /usr/i586-mingw32msvc/lib/ sudo cp -R src/include/omlet* /usr/i586-mingw32msvc/include/
      Fries:
      cd fries aclocal; libtoolize; autoconf; automake -a ./configure --host=mingw32 && make sudo cp src/libfries/.libs/libfries.a /usr/i586-mingw32msvc/lib/ sudo cp -R src/include/fries* /usr/i586-mingw32msvc/include/ sudo cp src/include/regexp-pcre++.h /usr/i586-mingw32msvc/include/
      Freeling
      export CFLAGS=-DPCRE_STATIC
      export CXXFLAGS=-DPCRE_STATIC  
      export LDFLAGS=-DPCRE_STATIC  
      cd freeling aclocal; libtoolize; autoconf; automake -a ./configure --disable-libDB --host=mingw32  
      (if at this point you get an error like: "checking fries.h usability... no" and "fries.h not found", the problem, can be caused by the headers cause those won't compile, please review those in order to get the right headers, basically replace "" with <> from the right places, in the includes) make (This will compile the code *.o but will fail in the linking process, to get the final dll and .exe will have to follow those instructions) If you need the analyzer.exe tool edit src/main/sample_analyzer/socket.h, in order to support winsock for the mingw:
      #ifdef WIN32 # include <winsock2.h> #else # include <sys/socket.h> # include <sys/param.h> # include <netinet/in.h> # include <arpa/inet.h> # include <netdb.h> #endif
      and this is the command to generate the morfo.dll :  
      cd src/libmorfo  
      # i586-mingw32msvc-g++ -shared -shared-libgcc -o morfo.dll -Wl,--add-stdcall-alias -Wl,--out-implib=libmorfo.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--whole-archive .libs/libmorfo.a -Wl,--no-whole-archive -ldb_cxx -lpcre -lfries -lomlet -lboost_filesystem-mt -lboost_system-mt -lws2_32  
      if you get an error with the property_map.hpp, this can be caused cause you have a different version. I solved the issue editing the files: disambiguator/ukb/common.h, disambiguator/ukb/kbGraph.h replacing this: -#include <boost/property_map.hpp>
      With this:
      +#include <boost/property_map/property_map.hpp>
      If you get an error related to pthread this can happen cause some crappy installations (ubuntu .deb package) don't include this as standard. Pelase visit :http://sourceware.org/pthreads-win32/ to get a valid version. cd src/APIs/java This will generate the skeleton of the java lib:  
      #swig -java -c++ -package morfo -outdir morfo -o libmorfo_wrap.cxx -I/usr/share/swig1.3/java -I/usr/share/swig1.3/std -I/usr/share/swig1.3 -I/mnt/old/src/freeling/freeling/APIs/morfo libmorfo_java.i and this will generate the morfo_java.dll  
      #i586-mingw32msvc-g++ -shared-libgcc -o morfo_java.dll libmorfo_wrap.cxx -Wl,--add-stdcall-alias -shared -Wl,--out-implib=morfo_java.dll.a -Wl,--export-all-symbols -Wl,--enable-auto-import -Wl,--no-whole-archive -lmorfo -ldb_cxx -lpcre -lws2_32 -I. -I/usr/local/java/include/ -I/usr/local/java/include/linux/  
      If you modify slightly the Makefile at APIs/java will get the same result wiht less effort. If you get problems with the pthread version can try removing the pthread include from iso2utf.h, it doesn't use it and will work. Finally if you are having problems wiht maco (nice in catalan ) you have to follow this note: ## There used to be problems when deleting "maco" objects from java. ## Maybe swig has fixed this... If you experience such problems, ## overwrite class morfo/maco.java with classes_aux/maco.java before ## packing the .jar this will generate the analyzer.exe  
      #i586-mingw32msvc-g++ -DPCRE_STATIC -O3 -Wall -DPCRE_STATIC -o analyzer.exe analyzer.o -L/usr/i586-mingw32msvc/lib -lmorfo -lfries -lomlet -lcfg+ -ldb_cxx -lpcrecpp -lpcre -lws2_32 -lboost_filesystem-mt
      Shared libgcc: If all modules are linked with -shared-libgcc, exceptions can be thrown across DLL boundaries.

      Finally The Binaries:

      Well maybe you just don't care about the process, don't have a linux machine to compile or simply don't have the time to dedicate, to compile this. Here you have a tested binary package that can use. http://dl.dropbox.com/u/1974426/freeling-2.2-mingw32.zip Those binaries were compiled in a VirtualBox with an Ubuntu10.10 installed and GCC 4.2.1-sjlj (mingw32-2) as the compiler.

      Conclusion:

      I think Freeling is a great tool, Luis and all the developers that have worked in that tool did a great job, and one demonstration of this good work is that the code can be cross compiled without changing the code. I like cross compiling and i enjoyed preparing this guide and getting the work done with the binaries for windows compiled in my linux box, but i have to recognize is more complicated and error prone than it should. I hope in the future we will be able to integrate the little changes in the original compiling process, maybe can get some ideas from pcre (make pcre.dll). Thanks for your time and attention, if you find errors please send the corrections to the development mailing list.

Comentarios

  1. Muchas gracias, me resulta muy util, para un proyecto que tengo en mente, la plataforma sobre la que trabajo con mis clientes es 99% Windows, asi es la vida....

    Juan Carlos Rodriguez O.
    Bogota, Colombia.
    quirogaco@gmail.com

    ResponderEliminar
  2. una pregunta como puedo compilar las librerias para python..en windows..

    Juan Carlos Rodriguez O.
    Bogota, Colombia.
    quirogaco@gmail.com

    ResponderEliminar
  3. Encontre este problema:

    D:\opensource\nlp\spanish\freeling\freeling-2.2-mingw32>bin\analyzer.exe -f es.c
    fg <mytext.txt
    terminate called after throwing an instance of 'std::logic_error'
    what(): basic_string::_S_construct NULL not valid

    This application has requested the Runtime to terminate it in an unusual way.
    Please contact the application's support team for more information.

    uan Carlos Rodriguez O.
    Bogota, Colombia.
    quirogaco@gmail.com

    ResponderEliminar
  4. en cuanto a lo de Python para win32,, no lo habia compilado porque no me habia hecho falta, intento compilarlo y te lo incluyo, no prometo fechas.

    en cuanto al error, no tengo claro el origen, en que casos se da? estas seguro de que el cfg y el fichero están bien?

    Saludos desde Pamplona.

    ResponderEliminar
  5. Gracias por tu respuesta, voy a revisar el archivo cfg.

    Juan Carlos Rodriguez O.
    Bogota, Colombia.
    quirogaco@gmail.com

    ResponderEliminar
  6. Ya revise era error de configuracion...funciona bien gracias..quedo pendiente de lo de python.

    para cuando puedas.

    Juan Carlos Rodriguez O.
    Bogota, Colombia.
    quirogaco@gmail.com

    ResponderEliminar
  7. Gracias por la compilacion y por los pasos. Ya intenté yo la compilacion con mingw bajo windows pero no pude generar la librería para java. Te invito a visitar www.textosign.es donde se hace uso de freeling como analizador morfosintactico para la traducción del lenguaje natural al lenguaje de sordos.

    ResponderEliminar
  8. rafaeljl: De nada. Muy interesante vuestro proyecto, lo tendre en cuenta. Inspirador.

    ResponderEliminar
  9. Hola Israel, muy interesante tu articulo y completo tu articulo. Te consulto por si puedes ayudarme ya que estoy con problemas al compilar la boost library.
    En cuanto al boot-jam dentro del directorio no tengo ningun bootstrap.sh, pero lo corri a traves del comando /build.sh --without-icu. Con esto genere el bjam.
    Luego con el bjam, al intentar correr la sentencia larga dentro del directorio boost, me falla ya que dice no contener un archivo user-config.jam.
    Tienes idea que puede estar pasando? Desde ya muchas gracias!

    ResponderEliminar
  10. Me agrada mucho que muestres esta guia.. pero yo eh estado intantando crear un jar para usarlo en linux.. hasta ahora solo eh conseguido hacerlo funcionar en consola.. pero al momento de gregarlo a un proyecto de netbeans falla al cargar morfo_java.. saludos!..

    ResponderEliminar
  11. me refiero a usarlo sin nada de win.. solo linux :D.. compilo todo bien.. el problema creo que es al crear el jar.. en el foro de la pagina de freeling muchos tienen el mismo problema... y yo soy uno de ellos saludos :p

    ResponderEliminar
  12. Hi,
    I am trying to test the sample provided in the user manual of freeling. I am using freeling 2.2 in windows. everything works properly but the grammar-dep.dat. I get an error with the grammar.lemastemp:

    GRAMMAR: Error opening file grammar.lemastemp

    the file is in the same folder than the grammar-dep.dat, so i don´t understand the error. I have tried to change the path in the rules using grammar.lemastemp wrtting it with the absolute path, but it gives me another error:

    for example, in this rule:
    nom-tmp-ms ==> NCMS*<"C:\\grammar.lemastemp">.
    I have tried it with two backslashes, without backslashes but I cannot make it work.
    I have written a post in the forum and they have referred to you.
    Kind regards

    ResponderEliminar
  13. Last distributed (binary) windows version of FREELING (Freeling 2.2 compiled with mingw32) produces bad output (first character omitted) tokenizing some Spanish words. Try, for example, to tokenize the following spanish UTF8 phrases:

    Çesaron los gritos.
    Ósculos para todos.
    Ídolos de barro.

    ResponderEliminar
  14. Por favor, alguien me puede decir cómo obtengo las librerias .dll para java??? He usado las .dll que estaban en el fichero y no funciona, me da el error:

    Exception in thread "main" java.lang.UnsatisfiedLinkError: freeling.libmorfo_javaJNI.new_maco_options(Ljava/lang/String;)J
    at freeling.libmorfo_javaJNI.new_maco_options(Native Method)
    at freeling.maco_options.(maco_options.java:197)

    ResponderEliminar
  15. Hola

    Estoy tratando el porting que has hecho para windows, enhorabuena, y muchas gracias por tu trabajo, creo que es fantástico.

    Estoy intentando integrar freeling con java.

    Cuando compilo, no tengo ningún problema, pero al ejecutar me sale el siguiente error:

    GRAMMAR: Error opening file grammar.lemastemp

    Por favor, podrías ayudarme???

    Gracias, Javier.

    ResponderEliminar
  16. Hola Israel, espero que siga atendiendo este post porque tengo un problema bastante serio y llevo muchos dias intentando solucionarlo sin exito. Use tu version cross-compiled de freeling y necesito usar el desambiguador. Para usar este módulo es necesario emplear dos herramientas que producen dos archivos necesarios para el procesamiento, las herramientas son compile_kb y convertdict. Estan como programas fuente de C en los archivos comprimidos, pero tu version no incluye (por lo que veo) ni estos ejecutables compilados para windows ni los archivos que genera. He intentando compilarlos pero tienen muchisimas dependencias, entre ellas estas:

    #include
    #include
    #include
    #include
    #include

    // Stuff for generating random numbers

    #include
    #include
    #include
    #include
    #include

    que deben pertenecer a la instalación de libboost. Me han comentado los desarrolladores de freeling que debe ser bastante complicado instalar libboost en Windows y que intentara realizar mi programa en linux, pero yo no tengo esa opción, y por eso recurrí a tu versión cross-compiled.
    Tienes acaso las utilidades compile_kb y convertdict en versión para Windows o en su defecto, los archivos que generan para poder utilizar el desambiguador??

    Estaría muy agradecida si pudieras contestarme. Muchisimas gracias de antemano y un saludo y enhorabuena por tu trabajo.

    ResponderEliminar
  17. Hola Esther,

    No tengo generados los ficheros que me comentas, tengo que volver a actualizarlo y compilarlo, pero estoy bastante liado y no me atrevo a prometerte cuando lo volveré a compilar. Seguro que lo publicare aquí.

    Saludos y gracias.

    ResponderEliminar
  18. Hola Israel

    Cuando gastas el freeling en una web, hay que hacer algo cada vez que lo usas, para destruir variables o algo, cerrarlo, etc.? Lo uso en un tomcat en windows, y cada vez que lo uso sale todo bien, pero se queda la web inestable y sale un error de java que dice que se ha produdido un crash fuera de la máquina virtual.

    gracias!!!

    ResponderEliminar
  19. Este comentario ha sido eliminado por el autor.

    ResponderEliminar
  20. Hi, I am working on a my project under windows xp/7; the analyzer of Freeling 2.2 works very well, but I have a litte question:
    If I want write a small code in C on windows, my compiler dont found the file regexp-pcre++.h .

    Can you help me?

    Maybe under windows it's not possible programming?

    Many Thanks

    #include
    #include
    #include
    #include

    #include " my path /freeling.h" <- or other file.h of Freeling

    //using namespace std;

    int main(int argc, char *argv[])
    {
    printf("hello, world");
    return EXIT_SUCCESS;
    }

    ResponderEliminar
  21. Thank you for useful instruction. I cannot open the instructions page at http://www.lsi.upc.edu/~nlp/freeling/index.php?option=com_wrapper&Itemid=63

    ResponderEliminar

Publicar un comentario

Entradas populares de este blog

Como intentan robar las mafias utilizando AutoScout24

Liferay 6.1 + SOLR 4 + Zookeeper = Massive scalability