Juha Palomäki

Quick instructions for enabling firewall on Ubuntu using Uncomplicated Firewall (ufw). The resulting rules are quite obvious. Incoming traffic is denied, except for ssh, http and https (these refer to /etc/services) and outgoing is allowed.

These policies do not affect ICMP. Incoming ICMP remains allowed as per default ufw settings.

sudo apt-get install ufw
sudo ufw allow ssh
sudo ufw allow http
sudo ufw allow https
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw enable

Quick install script for installing mongodb a service on Windows. Further instructions are available in official MongoDB docs.

Download appropriate version: http://www.mongodb.org/downloads
extract to your folder of choice, startup cmd.exe as Administrator and change to the MongDB folder.

REM -- Setup directories
set mongodir=%PROGRAMDATA%\mongodb
mkdir %mongodir%
mkdir %mongodir%\data
mongod.exe --dbpath %mongodir%\data --logpath %mongodir%\log.txt --install
REM -- Read the logfile, check install was ok
type %mongodir%\log.txt
REM -- Start the service
net start mongodb

If you run into “System error 193 has occurred. *** is not a valid Win32 application.” -error when starting the service, there is probably some error in the mongod.exe command line. The command like you use here is used when installing the service (you can check it via Windows Service Manager). For example omitting the .exe -extension or having some extra spaces in dir names can cause these problems.

Simple steps for getting new IP address via DHCP on Ubuntu (tested on Ubuntu 11.10, running under vmware with bridged network)

Sublime Text stores settings and packages in %APPDATA%\Sublime Text 2 folder by default. Quick search did not reveal any way of forcing it use some other location. In order to sync the settings by using Dropbox, the files should obviously reside inside the Dropbox folder (eg c:\users\juhap\Dropbox)

The following instructions are written for Sublime Text 2 (beta) and Windows 7. Some older versions of Windows might not have the mklink -command that is used to create a symbolic link.

1. Create a new folder inside your Dropbox folder, for example c:\users\juhap\Dropbox\apps\sublime
2. Move Packages, Installed Packages and Pristine packages from %APPDATA%\Sublime Text 2 to the new folder
3. Create symbolic links for the directories pointing from the Sublime data directory towards your Dropbox folder. In order to do this, you need start cmd.exe as Administrator. Navigate to %APPDATA%\Sublime Text 2 before executing the following commands:
   mklink /D "Installed Packages" "C:\Users\juhap\Dropbox\apps\sublime\Installed Packages"
mklink /D "Packages" "C:\Users\juhap\Dropbox\apps\sublime\Packages"
mklink /D "Pristine Packages" "C:\Users\juhap\Dropbox\apps\sublime\Pristine Packages"


And that’s it.

Encountered annoying MySQL problem when trying to dump database from a backup server.

MySQL kept complaining about
The user specified as a definer ('myuser'@'%') does not exist when using LOCK TABLES
when I ran mysqldump.

The database had been originally dumped from production server, then imported into a backup server. The production server and backup server have different sets of users. “myuser” only existed in production, not on backup. The problem was probably caused by some view that still referenced “myuser”.

A quick workaround was to create “myuser” on the backup server as well.

Loggly is a fairly new cloud base log management service. You can think about it as lightweight and hosted version of Splunk. I find it pretty interesting, especially because it provides pay-as-go licensing model which I believe suits better to SaaS business than Splunk’s traditional licenses. I also love the idea that Somebody Else is responsible for keeping the service up and running..

Loggly provides few options for getting the logs into their system. You can use either syslog (tcp or udp) or HTTP. They also provide (or at least link to) some Ruby scripts that can take care of monitoring existing log files. In our case we needed to collect the logs from multiple Linux servers running JBoss. One issue that made things a little bit more complicated was that for certain reasons we only wanted to make changes to the JBoss configuration. The provided Ruby scripts did not feel a good solution as they required ruby & rubygems to be installed (and our servers did not have those).

A logical option was to start investigating how to get logs directly from JBoss to Loggly. JBoss 5.1.0 uses Log4j by default. There were few options. Loggly provide links to various projects that provide log4j appenders that work with Loggly HTTP API. Instead of using those, I decided to tryout syslog. Log4j comes with syslog appender, but there is one major drawback associated with it. The standard appender does not support TCP, only UDP. And sending the logs as UDP packets over the internets did not feel good solution. In my use case it is acceptable to lose some logging for example when server is shutting down, but otherwise I want to make sure all messages end up in Loggly.

Luckily I found http://www.syslog4j.org/ – a pure-Java syslog implementation that has bee around for some time. And even better – it comes with built-in Log4J appender. Syslog4j is very comprehensive solution. It provides support for TCP and UDP among many other fine things and does not require any third party jars for basic operations.

Getting JBoss to work with syslog4j turned out to be quite a challenge although it is actually really simple when you know what you should do. The main problem is that if something goes wrong, you probably don’t get any errors. The logging just does not work. There are two options for doing this. You can either set the configuration inside war/ear or you can use the main JBoss configuration, which affects all applications. In my case I wanted to use the JBoss wide configuration as I don’t want to be modifying the wars when they are deployed to production env where Loggly is used.

Steps on Loggly:

1. Create a new syslog TCP input. Make a note of the server name (probably logs.loggly.com) and the port number (which varies) – you will need these when configuring JBoss.
2. Make sure the input is in discovery mode. Discovery mode means Loggly will accept inputs from all IP addresses. The input will automatically stay in discovery mode for about 15 mins after it has been created. After that it will only accept traffic from hosts is already knows about.

Steps on JBoss 5.1.0:

1. Download syslog4j binary jar from the website e.g. http://www.syslog4j.org/downloads/syslog4j-0.9.46-bin.jar and place it to $JBOSS_HOME/lib directory.
2. Modify the $JBOSS_HOME/bin/run.sh script. Locate line that says:
   JBOSS_BOOT_CLASSPATH="$runjar"
   Modify it to:
   JBOSS_BOOT_CLASSPATH="$JBOSS_HOME/lib/syslog4j-0.9.46-bin.jar:$JBOSS_HOME/lib/log4j-boot.jar:$runjar"
   NOTE: This seems to be the important thing to get right. I believe the order of jars above can be significant.
3. Modify the log4j configuration. This is located in $JBOSS_HOME/server//conf/jboss-log4j.xml
4. Add new entry for Syslog4j appender that logs into Loggly into jboss-log4j.xml

   <appender name="LOGGLY" class="org.productivity.java.syslog4j.impl.log4j.Syslog4jAppender">
           <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
           <param name="Protocol" value="tcp"/>
           <param name="Facility" value="user"/>
           <param name="Host" value="<YOUR LOGGLY HOST, probably logs.loggly.com"/>
           <param name="Port" value="<YOUR LOGGLY PORT>"/>
           <layout class="org.apache.log4j.PatternLayout">
                   <param name="ConversionPattern" value="%c (%t) %m%n"/>
           </layout>
      </appender>
   

   The conversion pattern is also special. You can’t pass timestamps to Loggly (at least not right now). The messages the current time automatically, so there is no need to add them to log messages. Syslog4j appender also includes the priority so that is not needed either.

5. Add references to the newly added appender. Locate this section at the end of the file and add reference to Loggly.

         <priority value="${jboss.server.log.threshold}"/>
         <appender-ref ref="LOGGLY" />
         <appender-ref ref="CONSOLE"/>
         <appender-ref ref="FILE"/>
   

After making these changes and restarting JBoss you should see some activity on your Loggly account.

Two things I haven’t yet investigated are the performance of this combination and how errors are handled.

The calls to Log4J are blocking by default. This means the application returns only after the message has been written to log file (or sent to syslog, in this case). This could slow down the application if log writing is taking time. Log4J comes with Async appender that partly solves this problem. Async appender maintains internal queue for log messages and a separate thread that takes care of calling the actual appender. This means the calls from application can return quickly, even if writing the log message takes time. If the internal queue becomes full, the calls to async appender either block or it starts discarding the messages.

A (not yet tested) example is below. If you try this, remember to change the reference at the end of Log4J configuration to ASYNC_LOGGLY (instead of LOGGLY).

   <appender name="ASYNC_LOGGLY" class="org.apache.log4j.AsyncAppender">
     <errorHandler class="org.jboss.logging.util.OnlyOnceErrorHandler"/>
     <!-- How many messages can be kept in the queue -->
     <param name="BufferSize" value="100000"/>
     <!-- If buffer becomes full, drop log messages-->
     <param name="Blocking" value="false" />
     <appender-ref ref="LOGGLY"/>
   </appender>