A.A.A.R. v2.2 with interactive dashboards and free analysis

Finally the date came!

Starting from the requests and the collaboration of some of you, the brand new A.A.A.R. v2.2 has been released. I would like to explicitly thank strategicfunctions.com for the contribution. As usual it has been an interesting experience.

The main new feature is obviously the interactive dashboard on audit trail.

But also the two brand new free analysis on audit trail and repository.

Last but not least, a brand new report on documents in sites (strongly requested from the community).

I have lots of new ideas to develop but what would you like to see more in the A.A.A.R.?

A.A.A.R.

Discover the documents stored into your Alfresco sites… interactively!

Thank to the requests of some A.A.A.R. users, a brand new report showing the documents stored in every Alfresco site, has been developed. The report is interactive in the way you see below.

Report12_menu

Below a preview of the report for a test case.

Report12

The report will be part of the incoming AAAR v2.2 but if you want to use it or customize it, you can download from github in the PostgreSql version or the MySql version.

Enjoy your analytics.

Francesco Corti

Alfresco 5.0.b CE – Something to fix with Solr4 using CMIS 1.1

bugThanking some of you, during the use of A.A.A.R. some problems appeared using Alfresco 5.0.b Community Edition. Below is described a simple use case using the recent release of the CMIS Input plugin for Pentaho Data Integration v5.2. Waiting for the solution and bugfix, thank you again for the submissions and contributions.

Starting from a vanilla installation of Ubuntu 14.04 LTS with Pentaho Data Integration v5.2 on board, the most recent release of the CMIS Input plugin v1.3 has been installed.

After this easy step, a simple transformation is developed to enquiry Alfresco 5.0.b Community Edition installed from scratch with Solr4. Below the screenshot representing the CMIS query and connection.

CMIS query

Executing the query, this is message we receive.

CMIS query error

And this is the catalina.out log.

Error log

Removing the ‘order by’ from the query, or the ‘or’ operator, the query runs correctly.  Unfortunately, trying to install Alfresco 5.0.b Community Edition with Solr1, the first startup fails and we are locked. Let’s wait for some good news from the next release.

Francesco Corti

CMIS Input plugin v1.3 for PDI v5.2… extract your data from your ECM.

PDI CMIS Input plugin

The change log page describes some more details but the most important feature is about the used of the Apache Chemistry libraries updated at the v0.12. This version is useful to support the CMIS 1.1 for Alfresco and more.

During the next hours the new version will be available in the Pentaho marketplace… stay tuned!

Francesco Corti

How to install Alfresco 5.0.a Community Edition on Ubuntu 14.04 LTS

alfresco logoAfter the development of my most viewed post sharing how to install Alfresco 4.2.c on Ubuntu 10.04 LTS (more than 35.000 views) and Windows Server 2008 R2 (more that 13.000 views), it’s time to update the tutorial to the newest major version of Alfresco: Alfresco Community Edition 5.0.a. Even if the 5-th version is quite different from the 4-th version, the installation process is more or less the same… but let’s describe exactly the differences. The operating system choosen for the tutorial is Ubuntu 14.04.01 LTS.

Differently from the other post, the tutorial is separated in two parts: the installation of the dependencies and the Alfresco installation. As we like and prefer, the installation is a step by step list of commands and tasks: simpler to understand, to do and to test. Hope you’ll be agree.

Part 1 – Installation of the dependencies

Starting from a vanilla installation of Ubuntu O.S. 14.04.01 LTS, let’s update the libraries and configurations.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get purge openjdk-\*

Now it’s time to add a new user called alfresco, we will use for the installation (password: alfresco).

adduser alfresco
Enter new UNIX password: alfresco

Starting from now we will work as alfresco user.

su – alfresco

JDK 1.7u67

Even if Alfresco 5.0.a is certified with Java1.7U60, we use Java1.7U67 (nothing serious will happen). You can download the package from Oracle Java SE Downloads. In our case the package is ‘jdk-7u67-linux-x64.tar.gz’.

sudo mkdir -p /opt/java

Unzip the package in it. In our case in the ‘/opt/java/jdk1.7.0_67’. Please remember to set the permits to the folder to ‘execute’ (use can use chmod command).

sudo nano /etc/profile.d/java.sh
export JAVA_HOME=/opt/alfresco/java/jdk1.7.0_67
export PATH=$PATH:$HOME/bin:$JAVA_HOME/bin

Save and exit.

java -version

ImageMagick

sudo apt-get install ghostscript imagemagick
convert --version

Using the command ‘whereis convert’, please check the result is ‘/usr/bin/convert’. If not, copy and use it in the ‘img.exe’ parameter into the alfresco-global.properties file.

FFMPeg

sudo add-apt-repository ppa:jon-severinsson/ffmpeg
sudo apt-get update
sudo apt-get install ffmpeg

SWFTools

sudo apt-get install libjpeg62 libgif4
wget http://archive.canonical.com/ubuntu/pool/partner/s/swftools/swftools_0.9.0-0ubuntu2_amd64.deb
chmod a+x swftools_0.9.0-0ubuntu2_amd64.deb
sudo dpkg -i swftools_0.9.0-0ubuntu2_amd64.deb

Using the command ‘whereis pdf2swf’, please check the result is ‘/usr/bin/pdf2swf’. If not, copy and use it in the ‘swf.exe’ parameter into the alfresco-global.properties file.

LibreOffice

sudo apt-get install libreoffice

Using the command ‘whereissoffice’, please check the result is ‘/usr/bin/soffice’. If not, copy and use it in the ooo.exe’ and ‘jodconverter.officeHome’ parameters into the alfresco-global.properties file.

PostgreSql

Even if Alfresco 5.0.a is certified with PostgreSql 9.2.4, we use PostgreSql 9.3.5 (nothing serious will happen).

sudo apt-get install postgresql postgresql-contrib
sudo passwd postgres
postgres

Now it’s time to prepare the (empty) database schema for Alfresco.

sudo -u postgres psql postgres
CREATE ROLE alfresco WITH PASSWORD 'alfresco' LOGIN;
CREATE DATABASE alfresco WITH OWNER alfresco;
<ctrl+d>
sudo -u alfresco psql alfresco
ALTER USER alfresco WITH PASSWORD 'alfresco';
<ctrl+d>

Tomcat

Even if Alfresco 5.0.a is certified with Tomcat 7.0.53, we use Tomcat 7.0.55 (nothing serious will happen). You can download the package from Apache Tomcat. In our case the package is ‘apache-tomcat-7.0.55.tar.gz’.

sudo mkdir -p /opt/alfresco
sudo chown alfresco:alfresco /opt/alfresco

Unzip the package in ‘/opt/alfresco’ renaming the apache-tomcat folder in ‘tomcat’.

sudo chown -R alfresco:alfresco /opt/alfresco/tomcat
/opt/alfresco/tomcat/bin/startup.sh
ps -ef | grep java

You can check the existence of the java process. Alternatively you can access to http://localhost:8080/ using your browser.

/opt/alfresco/tomcat/bin/shutdown.sh

Part 2 – Installation of Alfresco

Now that everything is ready, we can proceed to install Alfresco into the environment.

Alfresco 5.0.a

cp /opt/alfresco/tomcat/conf/catalina.properties /opt/alfresco/tomcat/conf/catalina.properties.orig
nano /opt/alfresco/tomcat/conf/catalina.properties

Set the ‘shared.loader’ parameter with the value described below.

shared.loader=${catalina.base}/shared/classes,${catalina.base}/shared/lib/*.jar

Save and exit.

cp /opt/alfresco/tomcat/conf/server.xml /opt/alfresco/tomcat/conf/server.xml.orig
nano /opt/alfresco/tomcat/conf/server.xml

Add ‘URIEncoding=”UTF-8″‘ to ‘<Connector port=”8080″ protocol=”HTTP/1.1″…’.

 nano /opt/alfresco/tomcat/conf/context.xml

Add the line below:

<Valve className="org.apache.catalina.authenticator.SSLAuthenticator" securePagesWithPragma="false" />
mkdir -p /opt/alfresco/tomcat/shared
mkdir -p /opt/alfresco/tomcat/shared/classes
mkdir -p /opt/alfresco/tomcat/shared/lib
mkdir -p /opt/alfresco/tomcat/endorsed

Download Alfresco Community Edition from Sourceforge. The package is named ‘alfresco-community-5.0.a.zip’. Unzip the package in a temporary folder and you will find a folder named ‘alfresco-community-5.0.a’.

cd .../alfresco-community-5.0.a
cp -R bin /opt/alfresco
cp -R web-server/endorsed/* /opt/alfresco/tomcat/endorsed
cp -R web-server/shared/* /opt/alfresco/tomcat/shared
cp -R web-server/lib/* /opt/alfresco/tomcat/lib
cp -R web-server/webapps/* /opt/alfresco/tomcat/webapps/

You can remove the folder named ‘alfresco-community-5.0.a’.

nano /opt/alfresco/start_oo.sh
#!/bin/sh -e

SOFFICE_ROOT=/usr/bin
"${SOFFICE_ROOT}/soffice" "--accept=socket,host=localhost,port=8100;urp;StarOffice.ServiceManager" --nologo --headless &

Save and exit.

chmod uga+x /opt/alfresco/start_oo.sh
/opt/alfresco/start_oo.sh
ps -ef | grep soffice

You can check the existence of the soffice process.

killall soffice.bin
nano /opt/alfresco/alfresco.sh
#!/bin/sh -e

# Start or stop Alfresco server

# Set the following to where Tomcat is installed
ALF_HOME=/opt/alfresco
cd "$ALF_HOME"
APPSERVER="${ALF_HOME}/tomcat"
export CATALINA_HOME="$APPSERVER"

# Set any default JVM values
export JAVA_OPTS='-Xms512m -Xmx768m -Xss768k -XX:MaxPermSize=256m -XX:NewSize=256m -server'
export JAVA_OPTS="${JAVA_OPTS} -Dalfresco.home=${ALF_HOME} -Dcom.sun.management.jmxremote"

if [ "$1" = "start" ]; then
 "${APPSERVER}/bin/startup.sh"
 if [ -r ./start_oo.sh ]; then
  "${ALF_HOME}/start_oo.sh"
 fi
elif [ "$1" = "stop" ]; then
 "${APPSERVER}/bin/shutdown.sh"
 killall -u alfresco java
 killall -u alfresco soffice.bin
fi
chmod uga+x /opt/alfresco/alfresco.sh
sudo nano /etc/init.d/alfresco
 #!/bin/sh -e

ALFRESCO_SCRIPT="/opt/alfresco/alfresco.sh"

if [ "$1" = "start" ]; then
 su - alfresco "${ALFRESCO_SCRIPT}" "start"
elif [ "$1" = "stop" ]; then
 su - alfresco "${ALFRESCO_SCRIPT}" "stop"
elif [ "$1" = "restart" ]; then
 su - alfresco "${ALFRESCO_SCRIPT}" "stop"
 su - alfresco "${ALFRESCO_SCRIPT}" "start"
else
 echo "Usage: /etc/init.d/alfresco [start|stop|restart]"
fi
sudo chmod uga+x /etc/init.d/alfresco
sudo chown alfresco:alfresco /etc/init.d/alfresco
mkdir /opt/alfresco/alf_data
cp /opt/alfresco/tomcat/shared/classes/alfresco-global.properties.sample /opt/alfresco/tomcat/shared/classes/alfresco-global.properties
nano /opt/alfresco/tomcat/shared/classes/alfresco-global.properties

Below the lines to replace or add to the properties file.

dir.root=/opt/alfresco/alf_data
...
db.username=alfresco
db.password=alfresco
...
# OpenOffice
ooo.exe=/usr/lib/libreoffice/program/soffice.bin
ooo.enabled=true
jodconverter.officeHome=/usr/lib/libreoffice
jodconverter.portNumbers=8100
jodconverter.enabled=true
# ImageMagick installation
img.root=/usr/share/doc/imagemagick
img.exe=/usr/bin/convert
# SWFTools exe
swf.exe=/usr/bin/pdf2swf
...
db.schema.update=true
...
db.driver=org.postgresql.Driver
db.url=jdbc:postgresql://localhost:5432/alfresco
...
index.recovery.mode=AUTO
...
authentication.chain=alfrescoNtlm1:alfrescoNtlm

Now everything is ready to start alfresco but, please, pay attention now. Check very carefully the log during the first run. Pay attention to the ERRORS and the reasons why the errors are thrown.

service alfresco start
tailf /opt/alfresco/tomcat/logs/catalina.out

For example, the lines below are caused by the missing configuration of FTP and CIFS. Enjoy!

2014-09-22 22:01:32,203 ERROR [org.alfresco.fileserver] [FTP Server] [FTP] FTP Socket error : java.net.BindException: Permission denied
...
2014-09-22 22:01:32,210 ERROR [org.alfresco.fileserver] [CIFS Server] [SMB] Server error : org.alfresco.jlan.server.config.InvalidConfigurationException: Error initializing TCP-IP SMB session handler, Permission denied

A.A.A.R. not showing buttons (SOLVED)

After the installation, if your AAAR is not showing Configure/Install/Use buttons, please update Pentaho with the latest “Community Graphics Generator” and “Community Dashboards Framework”. The solution is described here.

aaar_not_showing_buttons

AAAR_for_marketplaceI have received few contacts on that problem so I think would be useful to share the solution one time for all. Thank you to jservajean and kossel for sharing the experience.

The order of the Bee. Something is happening in the Alfresco community.

Order of the BeeSomething interesting is happening in the Alfresco’s community.

I’m not talking about the Alfresco Community (Edition) of the software but the Alfresco’s community composed by people and skilled professionals.

Few weeks ago a brand new organization has been created and is called Order of the Bee. Why that name? Everything was born in the past Alfresco Summit with a Community Keynote by Jeff Potts (here the complete speech).

In my opinion, and not only in my opinion, this is a good news for the (pure) open source developers and professionals. I personally decided to join the organization together with the AAAR solution but I feel like something more interesting will happen in the nearest future. 🙂

Below I would like to submit to your attention some key concepts to better understand this organization.

Solr doesn’t return more than 1,000 objects in Alfresco.

lucene_apacheOnce upon a time Alfresco used Apache Lucene as search engine….

This was great until you had particular needs like, for example, a long duration query or a query that retrieves a huge amount of objects. It was more than a year ago when I wrote a post talking how Alfresco retrieves 1,000 results maximum or query for a couple of minutes.

solrAs you can read in the post, the most suggested solution to the problem was to migrate the indexing engine to Apache Solr. At that time, Alfresco supported both the engines and considered Solr as its future.

Today Lucene and Solr are always supported and Solr is probably the most used, but regarding the same issue, probably something is coming back again.

>> https://issues.alfresco.com/jira/browse/ALF-20567(*) <<

As you can read from the JIRA issue, in Alfresco 4.2.e SOLR also returns a maximum of 1,000 results and to solve the issue is suggested to set the parameters below in the alfresco-global.properties file.

solr.query.maximumResultsFromUnlimitedQuery=10000
system.acl.maxPermissionChecks=10000

This could have a high impact on “big” queries or “long” queries so I would like to share this information with all of you to prevent problems or nights spent on the debugger. 😉

I hope this will help you.

Francesco Corti

(*) Thanks to Francesco Fornasari and Christian Tiralosi for the hint.

Yet another Alfresco Community upgrade tutorial: from 4.0.d to 4.2.f.

The task to upgrade Alfresco (Community or Enterprise) from a version to another more recent, has to follow a clear and precise path. In every case it is always a critical task and in some cases could be a serious problem for the Organizations (of course this is more critical for Community Editions). In some cases the only possible solution is an Alfresco-to-Alfresco migration instead of an upgrade… but this is another scenario.

In this tutorial is described a step-by-step approach to an upgrade from an Alfresco Community Edition v4.0.d to v4.2.f in one only upgrade step. In every case, even if the involved versions are different, the approach is always the same discussed here.

Needless to say: I am not responsible for any damage that may happen after following the given instructions, which hopefully will not happen.

The (only) correct approach

Before starting I would like to share the (only) correct approach: please remember that the upgrade process for the Alfresco Community Editions is tested (and not guarantee) for the closest versions (for Alfresco Enterprise you can take a look here). This means that the only path you can follow to upgrade a very old version to a recent one is always to develop multiple upgrades.

For example, if you come from the v4.0.d and want to go to the recent v5.0.a, it’s only written in the stars if the direct upgrade will work. The most verified approach is to develop the upgrade process with the steps described below:
– Upgrade from v4.0.d to v4.0.e,
– Upgrade from v4.0.e to v4.2.a,
– Upgrade from v4.2.a to v4.2.b,
– Upgrade from v4.2.b to v4.2.c,
– Upgrade from v4.2.c to v4.2.d,
– Upgrade from v4.2.d to v4.2.e,
– Upgrade from v4.2.e to v4.2.f,
– Upgrade from v4.2.f to v5.0.a.

You can take your own risks “jumping” some steps, and in some cases it would work, but nothing is garantee in every case. In this tutorial I decided to take a reasonable risk, often discussed in the forums and tutorials, and “jump” with a single upgrade process.

Preparing the upgrade

To develop the upgrade I need the Alfresco backup of my v4.0.e production installation. If you don’t know what is an Alfresco backup and how to obtain it, I strongly recommend to take a look here.

In this tutorial I choose to define a brand new server with the recent Alfresco installation (in our case the v4.2.f) but you could choose to use the same server. Of course, in this case, the task is even more critical and the steps are the same but developed in different folders from the “old” version of Alfresco.

The new Alfresco installation

As introduced before, in this tutorial I work in a vanilla server with Ubuntu 14.04 LTS on board. In the server is installed Oracle Java v1.7.60u, always installed as described here.

To install Alfresco you can follow this tutorial even if it describes one specific version (the installation steps don’t change too much). Alternatively you can choose to install it using the easier wizard. In every case you will install the target version of Alfresco, in our case: Alfresco Community v4.2.f.

For the purpose of the post, the way you use to install Alfresco is not relevant but remember that it will be your brand new server, so it’s always suggested to have the most robust and stable one. 😉

If you have some customizations (custom models, behaviors, actions or something else) not it’s time to install them in the new server. The task is always the same: stop alfresco, deploy the customizzations in the way you always do (AMP, maven, manually) and start Alfresco again.

As final step, it is always suggested to switch off the indexing. In our case we suppose to use Solr but with Lucene it will be the same. To develop the task, please follow the steps below:

cd <alfresco>
./alfresco.sh stop
nano tomcat/shared/classes/alfresco-global.properties

 ...
 index.subsystem.name=noindex
 #solr.port.ssl=8443 (comment it)
 ...

Save and exit.

Database restore

Now it’s time to restore the alfresco database from the backup. To do it, please be sure that PostgreSql (or the database you use) is running. If you installed the Alfresco with the wizard you can use the command below.

./alfresco.sh start postgresql

To delete the current Alfresco’s database use the commands below.

cd <postgresql>/bin
./psql -h localhost -U postgres -d postgres

  ...
  DROP DATABASE alfresco;
  CREATE DATABASE alfresco WITH owner = alfresco;
  \q
  ...

To restore the database dump you can use:

./pg_restore -h localhost -U postgres -d alfresco <file.dump>

Filesystem restore

Once the database is restored you have to restore the documents on the file system from the backup.

cd <alfresco>/alf_data
rm -rf contentstore
rm -rf contentstore.deleted

Now it’s time to copy the ‘contentstore’ and ‘contentstore.deleted’ folders form the backup, directly in the ‘alf_data’.

Can’t you see the indexes are not restored? If possible it’s always preferrable to rebuild the indexes from scratch. In the other cases we suggest to restore them from the backup, hoping nothing changed in the structure. 🙂

Alfresco bootstrap

Now everything is ready to start alfresco again.

cd <alfresco>
./alfresco.sh start
tailf tomcat/logs/catalina.out

You will see that the starting process is updating the database and everything is necessary to upgrade the system. Errors or problems will be listed here…

Indexes rebuild

As you read before, the Alfresco update has been without the indexes.
Now it’s time to rebuild them following what you read here.

./alfresco.sh stop
nano <alfresco>/tomcat/shared/classes/alfresco-global.properties

  ...
  index.subsystem.name=solr
  solr.port.ssl=8443
  ...

cd <alfresco>/alf_data/solr
rm -rf workspace/SpacesStore/*
rm -rf archive/SpacesStore/*
rm -rf workspace-SpacesStore/alfrescoModels/*
rm -rf archive-SpacesStore/alfrescoModels/*
cd <alfresco>
./alfresco.sh start

Enjoy your brand new Alfresco installation…

Francesco Corti

Alfresco roadmap for the next 12 months

roadmapAfter some requests from some users, the new Alfresco roadmap has been released in the official wiki. This roadmap doesn’t seems to be like the others of the past.

I read that the amount of topics are less than the past. By the way, each topic seems to be more detailed and “complete” (in the past most of the items were less specific than this). Comparing with the past roadmaps I can read a lot of “Enterprise only” in some important new features.

Have your own opinion reading the complete roadmap below.

https://wiki.alfresco.com/wiki/Product_Roadmap

Francesco Corti