Tag error: <txp:lxs_log_counter num_old_hits="10000" /> ->  Textpattern Warning: tag does not exist while_parsing_page_form: default, default
EXT. RANDOM ODDNESS - SCREENWRITING AND OTHER INSANITIES - Hit tracking plugins for Textpattern.
RANDOMODDNESS

Hit tracking plugins for Textpattern.

Saturday · September 03, 2005 · 02:49 PM

Welcome to version 0.9.07 of my hit tracking code! Written for Textpattern 4.0 (with added 404 functionality if you’re running Textpattern 4.0.1), these plugins log visitors and allow you to track information. Feature-rich and comprehensive, it’s designed to work and look like an integral part of Textpattern. These are also my first plugins, so be nice.

Consider this “alpha testing” – I’m interested in thoughts on the code as well as bugs and feature requests. All data that’s captured using this version will be deleted when you upgrade to a future version.

One plugin was good, two were great – now there’s three! This caused a name change, sorry about that. The biggest problem is the tag renaming – you’ll have to modify your pages and forms to reflect the new naming convention. I won’t change names again.

This version is intended to replace existing versions, so please upgrade even though you’ll lose the old tracking data. The next release will be a feature complete “beta” version and will also cause you to lose tracking data. once things are beta, I won’t change the database design and will be forwards compatible from there on out.

Be sure to check out the full installation notes below.

Features

  • new Country and city tracking added. This works very well, although it slows things down a little bit since it requires a lookup from a database on my server. See the tags below for displaying the country and city of your visitors!
  • improved 404 errors get forwarded to a web address you’ve specified, by default your homepage, but could be another site or a specific section on your site (and each 404 can be redirected to different locations). Combined with a few tags (see the More… link below), this is extremely powerful – including “automated searching” for hits that came from a known search engine. This requires Textpattern version 4.0.1.
  • new The plugin tracks file downloads and there’s a new chart on the View Details screen that lists the top 10 file downloads. Not perfect, since it doesn’t tell if a file transfer was aborted, but a nice way to see what people are clicking on.
  • new Auto-upgrade of the plugins! The “Configurator” has a link that will appear when the next version is released! No need to download future releases, unzip them, paste the encoded text into a form, etc, etc.
  • improved Unique Hit management is much better in this release. The new code allows tracking of the “Average Page View Time” (see View Details screen). Also removed ridiculously high numbers meaning the user went to your page, ate lunch, then came back and clicked something else. There’s now a ‘-’ flag for the first hit a user makes (counted as a “Unique hits”), then the ‘+’ flag for all other hits.
  • new A rudimentary mechanism for adding icons next to search engine hits, country flags, etc is available. Still more graphics to come, but it really enhances the view.
  • new Hit categorization allows you let the system know about unknown search engine, OS, Spammer, etc. There’s an update feature to synchronize any changes you make to a central server so others can share your configuration.
  • new “Panel layout” (located in the “Configurator” plugin) allows you to check off the charts you want to view and then select a placement, using radio buttons. This became necessary when a ton of new charts were added.
  • Detailed view displays a synopsis of things like duplicate hits, your hits, and number of 404s. There’s charts for listing the top sections, articles, search phrases, etc. Also shows percentage of browser/os/spider hits/countries to the site.
  • What’s captured is configurable. For speed reasons, several things can be turned off – like DNS lookup, and getting info about a browser/os/etc (which require a database hit). I’ve concentrated on speeding up hit_logger, the code that logs the hits.
  • To get rid of clutter on the list of hits on hit_viewer, things like search engines and spiders can be removed from view. Viewing hits isn’t as optimized as logging them, I concentrated more on features.
  • If your website visitor previously used a comment form and filled in a name, they’re considered a “friend” and their name can be displayed instead of an IP address or something that was looked up using DNS.
  • Auto config of the database. It’s pretty solid, even allows uninstalling/totally removing the tables I’ve added. The upgrade possibilities are also built in for future versions. It’s also extremely robust when only one of the two plugins are installed.
  • Works alongside the standard Textpattern logging (you can turn off the original logging to speed things up). Can also be disabled with an option on the Textpattern preferences tab. Combined with the previous feature, it’s unobtrusive and easy to remove.
  • Flags show the type of link and can be clicked to hide that type of link. For example, there’s a question mark next to a search engine hit – click and they’re not displayed. The header of that column turns into a link to reenable viewing search engine hits.
  • Full pagination of the hits that have been recorded along with variable viewing sizes. Just like standard Textpattern screens.

Known Issues with 0.9.07

Not many. There are several features to be added (see the More… link below) and some features I listed here might not work fully.

Installation

Upgrades If you’re upgrading, you need to go your “extensions:logger prefs” page (later alpha releases called this “extensions:logger flags”). Click the uninstall link at the bottom of the page to remove all database information. Next go to the “admin:plugins” page and uninstall the plugins from Textpattern. Follow the “Fresh Installs” directions.

Fresh Installs Install the three plugins using the “admin:plugins” form, enable them, and then view “admin:logs”. You’ll see some notices about missing pieces during the install, but once you view all of the pages everything should be fine.

Now just sit tight and watch the hits roll in.

Note that all hit information that’s collected with this alpha version will be removed when the next version of these plugins are installed.

The plugins (ver 0.9.07) are available here

Click the more button and read further for instructions on how things work, including configuration details and tags that are available.

Please.

Using lxs_log and Display options

If you go to the “admin:log” screen, you’ll see lxs_log, which displays a list of page hits – by default, every hit is viewed. The value in the “Who” column is the person that viewed the page and will be bold if this is the first time they’ve viewed your page. The “Page” column shows what they looked at on your site (and will be bold for 404’s). The “Referrer” will be bold unless it’s a search engine.

If you want to stop viewing certain types of hits, you can use the preferences screen (located under the Extensions tab, it’s the “lxs flags” page). An easier way is to use the flag values that are in the last column of the lxs_log screen.

The flags column has a header of ?I=*+#X^ and each character representing a type of page hit – search engines, yourself, file downloads, search spiders, unique/duplicate hits, spammers, 404’s, and redirects. As these types of hits are encountered, the appropriate flag will be put in this column.

If you scan through your list of hits, you’ll probably see one with the ‘I’, meaning you viewed the page. You’ll see that this letter is a link – click it to remove your own hits from view. The header of the flag column will change to reflect the change as well and you can click on the ‘I’ in the header to re-enable viewing your own hits. Hovering over these links will give you clues on what will happen if you click them.

Note that some hits will have multiple flags and won’t be displayed if any of the flags are clicked to hide them from view.

Once you get used to the meaning of these flags, you can turn on ‘Advanced View’ in the “extensions:lxs flags” screen. When enabled, the Advanced View will change the pop-up help strings to be more informative. The question mark will show the search string, ‘*’ will show the name of the spider, ‘+’ will show how long they viewed the page, etc.

The last configuration parameter for the view is ‘Shorten Strings’, which truncates some of the longer fields on the log view. Sometimes the reverse DNS of an IP address will return a long string or the full path and page that they viewed is so long that things will wrap in a weird way.

Like the flags, there’s a shortcut for disabling the Shorten Strings feature – it’s the three dots next to a string that was truncated. Hovering over the dots will pop up the full value of these fields and clicking the link will change the value of this parameter. Again, the header of the shortened columns will be modified with a shortcut for shortening the strings.

There’s a rudimentary mechanism for adding icons next to search engines, country flags, etc to the pages – enable the “Use Icons” flag.

Other Flags

The “extensions:lxs flags” tab has parameters divided into two sections – the ones we covered modify the Display and the rest will speed up viewing your site. To reduce the render time of a page, you can disable some options that cause extra database hits, like looking up browser or operating system information.

If you look through the Display parameters on the “extension:lxs flags” page, you’ll see that they are all enabled by default. The system will track as much detailed information as it can. Setting most of these parameters will speed things up by preventing a database look up. There are two exceptions:

  • Track Forum Hits – When your site is viewed from a link on the Textpattern forums, lxs_log will attempt to figure out the topic of the post that the user was viewing. This is done by querying the central server, which can be slow.
  • Track Location Info – The location of the viewer can be guessed by looking up some information based on their IP address. This requires a query of a central server, which performs some sophisticated database checks and returns the appropriate info.
  • DNS Lookup – IP addressed can be mapped into a human readable string that makes it easier to see where hits are coming from and also see what users are visiting your site. This involves some checking of database servers and can be quite slow.
  • Track View Time – When the user visits a page, information is recorded. If they visit a second page on your site, then the amount of time spent on the previous page is stored, requiring an extra database access.

There is also a parameter on the standard Textpattern screens that you can use to make things run faster. If you select the “admin:preferences” tab you can change the “Logging” parameter to be “none” to shut off the default Textpattern logging code. Note that you can also set the lxs_plugins_disabled flag, but why would you want to do that?

lxs configurator

This sophisticated set of pages provides several

Note that the hit_viewer uses the ‘Expire logs after’ value to clean up old database information. Set this value to the number of months that you want to store in the system. This value defaults to seven months.

lxs_log Tags

There are several tags that work in conjunction with the lxs_log plugin to display hit information or to allow conditional code to selectively handle items on your site. Roughly the tags are broken down into the following groups:

Informational

These display a string or “nothing” if the data is unknown. Some of these have conditional tags as well.

  • txp:lxs_log_counter – shows the number of hits to your site. If you want to replace an existing counter, put the number of hits that the old counter logged into the tag:
  • txp:lxs_log_remote – this is a string that identifies the name of the person (if they entered any comments on your site), the host name (if the dns lookup worked), or the IP address.
  • txp:lxs_log_browser – shows the name of the browser that is currently viewing the page.
  • txp:lxs_log_browser_ver – shows the version of the browser that is currently viewing the page. Note that this is often unknown.
  • txp:lxs_log_os – display the name of the OS that the viewer is running.
  • txp:lxs_log_os_ver – display the version of the OS that the viewer is running. Note that this is often unknown.
  • txp:lxs_log_search_phrase – output the search phrase (if any) that brought the user to the page. Note that this might not be the full search string – short words are removed and the remaining words are alphabetized.
  • txp:lxs_log_search_engine – display the name of the search engine that brought the user to this page.
  • txp:lxs_log_referring_host – the website that linked to this page.
  • txp:lxs_log_referring_uri – the page on the website that is linked to this page.
  • txp:lxs_log_location – this string is an attempt to identify the city and country that the person that is visiting your site. It might also contain state or territory information. See “lxs_log_known_location” below for more.
  • txp:lxs_log_country_code – the two character country code of the person visiting your site. See “lxs_log_known_location” below for more.

Conditional

These work with the standard txp:else logic to give you the ability to selectively display messages for the visitor.

  • txp:lxs_log_if_404 – condition is true if the user went to a page that didn’t exist on your site – they are forwarded to the current page and this flag is set.
  • txp:lxs_log_if_search – condition is true if the viewer came here from a search page (note it’s also true for search hits that resulted in a 404).
  • txp:lxs_log_if_own – condition is true if you were the person viewing the page. This could be useful for adding “edit” links to posts.
  • txp:lxs_log_if_spider – condition is true if the page was requested by a spider or robot. If you want to redirect or serve a special version of your page for automated web programs, this would be useful.
  • txp:lxs_log_if_duplicate – condition is true if the person has previously viewed a page on your website. This won’t work if the visitor has cookies disabled and only counts traffic since they started their web browser.
  • txp:lxs_log_if_spammer – condition is true if the page was requested from a spammer.
  • txp:lxs_log_if_referrer – condition is true if the referring host field is set.
  • txp:lxs_log_known_location – condition is true if the location of the visitor was retrieved. The system has a 75% location rate, some visitors will fail and a country code of ‘??’ is produced.

You should be able to add the following code to your page to handle 404 error conditions, along with automating a search for people that came from a search engine:

lists

There are other tags planned – Top/Last xx Search Phrases, Top/Last xx Search Words, Top/Last xx Referrers, Top/Last xx Hits, Top/Last xx Posts, etc.

Examples

The tags can be used in many different combinations. Here are a couple examples to show how they might be used. First is a bit of code that would work well on the default 404 page that your users will get redirected to.

<txp:lxs_log_if_404>
  <txp:lxs_log_if_search>
    <txp:lxs_log_search_engine /> : <txp:lxs_log_search_phrase />
    <p>Sorry, the page you were looking for wasn't found.  Here's some
    pages that might be close to what you were searching for.</p>
  <txp:else />
    <p>Sorry that the page that you were looking for couldn't be found.
    Feel free to poke around and see if you can find what you were
    looking for.</p>
    <txp:lxs_log_if_referrer>
      <p>You might also tell <txp:lxs_log_referring_host /> to fix the
      link to this site (from page <txp:lxs_log_referring_uri />).</p>
    </txp:lxs_log_if_referrer>
  </txp:lxs_log_if_search>
</txp:lxs_log_if_404>

If a 404 is encountered and the remote page was a search engine, we can display the search phrase. The plugins will automatically invoke the internal Textpattern search, so articles matching the search user’s search phrase will be displayed. If the link was not from a search engine, the user is told that the page wasn’t located and asks them to let the referring site know that the link is no longer valid.

Another interesting trick is to display the users location. The following will check to see if the location is known and displays the country code and the detailed location information as well.

<txp:lxs_log_if_known_location>
  <p>You're from <txp:lxs_log_country_code /> :
     <txp:lxs_log_location />!</p>
<txp:else />
  <p>I don't know you?!</p>
</txp:lxs_log_if_known_location>

Up Next

The next release will be feature complete, with the following functionality added:

  • Categorize Hits – still a bit of work to do with this.
  • Control the look/CSS of the Log and Details screens.
  • Full Internationalization, with downloadable language files.
  • Documentation and help information everywhere.
  • Detailed client info – resolution, is JavaScript enabled, etc.
  • Added images where possible.
  • More charts – weekly hits and monthly hits.
  • Speed up the main “View Log” page, in fact speed everywhere.
  • More tags! More options!