Non-apache love

As a small heads up for those of you who don’t use Apache, I’d like to bring your attention to our library of rewrite rules. We are often asked by users for help setting up URL Rewriting (which The Bug Genie 3 requires) with their server, and this post is there to point out what documentation we have.

Just right now, documentation for IIS 7 and later was added, meaning we now cover the most popular platforms. However are work is not yet done, with documentation for IIS 6 still unwritten (IIS 6 docs now out!), and apparently issues with the nginx one; on top of no doubt other servers that you wonderful users want to use The Bug Genie on (let us know if there is one you need!) Regardless, we currently support:

  • Apache (auto-generated by the installer)
  • IIS 6 (Windows Server 2003 and 2003 R2)
  • IIS 7 and later (so Vista/WS2008 and Windows 7/WS2008R2)
  • Nginx
  • Lighttpd

Further documentation, as well as the above (except Apache) can always be found in the Recipes section of our documentation.

More database encoding (or: how not to have fun with UTF-8)

zegenie posted a short while ago regarding encoding trouble with The Bug Genie, especially those of you who use Greek or Russian alphabets. The good news is we have made some progress on this topic!

The root of the problem is to do with how The Bug Genie connects to the database. Each component has different encoding:

  • Database: whatever you created it as (probably latin1)
  • Tables and columns: UTF-8 (we create them as that)
  • The connection itself: could be anything

The last of these is the crux of the problem. On my system (as I am from Britain), the connection is latin1. This is fine for me, as I don’t need any unicode characters. However, for those of you who do, this means that unicode data is inserted into a unicode table, but is converted to something else (such as latin1) in the process.

The net result is that it will look right on the screen, but in the database it is garbage. This is fine up to the point where you start trying to do stuff with it, as there is a potential for problems to occur with JSON and other unicode-specific things, as what is in the database is not unicode (it is infact latin1 or whatever)!

An example of this is to copy some Russian text into both the Title and Issue Description fields. It will appear correctly in the Issue Description but not in the Title (it will claim it is empty).

This is easily fixed by setting the connection to use UTF-8, by placing this call on line 539 of core/B2DB/classes/B2DB.class.php:

self::getDBLink()->query('SET NAMES UTF8');

Now, all unicode data will be stored correctly in the database. However, this now results in another problem!

Any mangled Unicode data stored in the database as latin1 will be outputted as stored in the database (as no conversion is going on). This means that the mangled text in the database will be rendered to the screen. Luckily this is easily resolved by outputting a dump of the database in a latin1 connection, and then reimporting in a utf-8 connection (the nice Unicode output is obtained on the latin1 export, not the mangle that you get if you exported as UTF-8). The following code will do this, but don’t do this unless you put in the above code change:

mysqldump -h localhost --user=root -p --default-character-set=latin1 -c --insert-ignore --skip-set-charset -r dump.sql thebuggenie
mysql --user=root -p --execute="DROP DATABASE thebuggenie; CREATE DATABASE thebuggenie CHARACTER SET utf8 COLLATE utf8_general_ci;"
mysql --user=root --max_allowed_packet=16M -p --default-character-set=utf8 thebuggenie < dump.sql

We will be continuing to work on these encoding issues for The Bug Genie 3.2.

Debugging The Bug Genie

After talking to many of you, it seems like your missing some good debugging features in The Bug Genie to try and figure out how to solve issues when things go wrong.

The Bug Genie ships in “normal mode”, which means only some of the logging feature and none of our debugging features are enabled. You can see the log in action if things go wrong – it will be shown below the “please report this issue” button. In addition to this, we also have a debugging mode, available if you switch it on inside the TBGContext class. If you’re familiar with PHP, this file is located in core/classes/TBGContext.class.php, and the property you want to change is located around line 34, and called $debug_mode. If you change its value to true, all your pages will now show the debug bar at the bottom.

The Bug Genie debug bar

The debug bar gives you information about:

  • The current route: The Bug Genie uses a routing-enabled MVC framework. The debug bar will tell you which route is currently being accessed
  • How long the page took to load: This includes several other timing details, like how long each template and template partial took to load. Click the time to see more information
  • The current scope: The Bug Genie is built to run on multiple servers in multiple “scopes”, where each scopes lives separate from eachother. This feature has not been completed yet, and as such there is no way to use this yet. It is, however shown in the debug bar.
  • Database information: The database information area shows you the number of database queries performed in this request, including how long it took. Clicking the database information area brings up the SQL log, which lists all SQL statements, including how long each statement took. The statements are numbered, so you can find them again in the full log.
  • Full log: To the far right is the “toggle log messages” link, which will toggle the complete log when clicked. This log includes all information messages, notices and warnings generated by The Bug Genie, including timing information, potential errors and detailed information about the system as it runs.

As you can see, there is much information available for debugging The Bug Genie should something go wrong. We’re using these features all the time, and are continually improving them. One of the things we will add is information about the current configuration in use, which can help us find and solve even more issues.

Do you have comments or suggestions for how we can improve? Let us know!

PHP optimizers and The Bug Genie – revisited

As we are seeing more issues with php optimizers interfering with the core functionality of The Bug Genie, we’d like to post a little clarification.

First of all, we do not have an issue with optimizers. They exist for a reason, and we will do our best to not have our code break when optimizers are used – for whatever reason issues may occur ( phpinfo() can confirm if you are using an optimizer – look for “eaccellerator” or “optimizer”).

However – and this is important – not all optimization techniques are valid, and not all optimization techniques are non-intrusive. One of the techniques used by php optimizers (e.g. eaccellerator) is to remove php docblocks before the php compilation step. This optimization “feature” is highly controversial, and it is intruding on a core php feature – php docblocks are a part of the php language. Removing php docblocks will cause applications to malfunction.

There are several php projects other than The Bug Genie already relying on this php language feature (the ability to read php docblocks at runtime):

  • Doctrine 2
  • Zend Framework
  • PHPUnit

To read more about this “optimization” technique controversy, I recommend having a look at these two links:
http://wildlyinaccurate.com/eaccelerator-and-doctrine-2/
http://blog.feryn.eu/2010/12/zend_soap_autodiscover-eaccelerator-causes-trouble/

The last blog post also contains information on how to disable this particular eaccellerator “feature” for certain files, and how to disable eaccellerator php docblock removal completely.

Unfortunately, we cannot do anything about this issue, as it is out of our control.

Version 3.0 issues and solutions

There have been a few issues reported with the installation and usage of the 3.0 version released recently. Most of the issues that have not been related to an issue in The Bug Genie itself relates to one of the two following system issues:

  • Outdated pcre (perl-compatible regular expressions) libraries
    On most platforms, PHP does not ship with its own set of pcre libraries, but will use system libraries where available. We’ve seen a number of issues where users trying to install The Bug Genie 3.0 on systems with outdated pcre libraries are experiencing problems with core functionality. These issues will be triggered by the regular expression searches in geshi (our included syntax highlighter) and in our own wiki parser. If you see errors regarding regular expressions, make sure you are using pcre libraries version >= 8.0 (f.ex.: some CentOS and RedHat server seems to ship with 2006 version 6.x, which will not work).
  • PHP optimizers
    Some servers are running PHP with optimizer modules such as the zend optimizer. One of the optimizing techniques used by the zend optimizer is to remove comments inside php files before they are compiled and ran on the server. Because The Bug Genie depends on docblock metadata for some operations, this kind of optimization will cause The Bug Genie to malfunction. This error can be noticed if you get error messages about missing class definition for foreign properties. The error also sometimes manifests itself as errors about trying to call methods on non-objects.

None of these errors are something we can fix. If you’re experiencing weird errors like the ones above, please try and disable any php optimizers running on your server, and / or update your pcre libraries, then try again.

Tip: Trouble installing on Windows?

A common issue with installing The Bug Genie on Windows is PHP can’t connect to the database, it just sits there for about 30 seconds and then gives up. If you have this problem, and your database is hosted on ‘localhost’, try using ‘127.0.0.1’ instead, and see if that works.

As a plus, if you use IIS, you will get a 404 error if there is a database connection error. This will be fixed in 2.1.2, but until then you can check the error by removing line 79 in install.php and trying again!