dp.cx blog

I've been doing some contract work over the last few weeks and brought back to life a system that I'd used when working at an agency: Bugwarrior, Taskwarrior, and Timewarrior. It requires simple applications that are written with an awesome amount of power.

Taskwarrior is the system I've been using off and on for a few years now that allows me to keep track of my task list. It's extremely flexible, scriptable, and powerful. About the only downside I know of is that there's no iOS app that I can use to interact with it (there are plenty of web apps; but since Taskwarrior is designed to work offline first, it would be nice if there were an app that I could use that was designed similarly). It supports rudimentary time tracking (task start/task stop), but is vastly outshone in that regard by Timewarrior.

Bugwarrior is a really good system that allows you to integrate many of the bug/issue tracking systems with Taskwarrior. It's a Python codebase that interacts with the APIs of systems like Github, Gitlab, Trello, and Jira (and many more), pulls down issues that are in your repositories (in the case of Github/Gitlab) or that are assigned to you (in the case of others) and imports them in to Taskwarrior. Bugwarrior, like Taskwarrior, is extremely flexible and powerful.

Timewarrior is a time tracking system that is built on top of Taskwarrior (are you beginning to see the theme here?) that can be used to track time on tasks. With custom reporting capabilities via scripts, it's really an amazing tool if tracking time is something you need to do.

Taskwarrior is the workhorse of this entire setup - it's hooks system allow you to set up other events that should happen when you take a specific action - for example, if you wanted to keep an audit log of all modifications to your tasks, you could use hooks to create this (Taskwarrior already does this in the undo.data file, but that's not for this post).

Timewarrior actually comes with a hook script out of the box that works with Taskwarrior. It allows you to use the built-in time tracking commands from Taskwarrior, but causes Timewarrior to also track the time - giving you all of the time tracking support your heart could desire.

Next, add in Bugwarrior. Bugwarrior will automatically create tasks in Taskwarrior for you based on the bug/issue trackers, allowing you to easily see them in your local environment. Bugwarrior can also pull comments on those bugs/issues and store them as annotations to the task, ensuring that the history of everything is available within Taskwarrior as well.

Finally, bring in Timewarrior. Yes, you could use Timewarrior on its own (timew start Write blog post would manually start a task, though it has no connection to Taskwarrior's task information), and in reality, the hook that Timewarrior provides is fully disconnected from Taskwarrior's task as well - you get the title, tags, and project. The problem here is if your bug/issue is renamed, which for me, was happening quite often.

Because of this issue (and a couple of other niceties), I modified the provided hook to look like this:

#!/usr/bin/env python3

## Note, this script lives in ~/.task/hooks
## as on-modify.timewarrior and needs to be executable

from __future__ import print_function

import json
import subprocess
import sys

try:
    input_stream = sys.stdin.buffer
except AttributeError:
    input_stream = sys.stdin

old = json.loads(input_stream.readline().decode("utf-8", errors="replace"))
new = json.loads(input_stream.readline().decode("utf-8", errors="replace"))
print(json.dumps(new))

def get_timewarrior_entry(json_obj):
    return 'uuid:' + json_obj['uuid']

start_or_stop = ''

if 'start' in new and 'start' not in old:
    start_or_stop = 'start'
elif ('start' not in new or 'end' in new) and 'start' in old:
    start_or_stop = 'stop'

if start_or_stop:
    timewarrior_entry_text = get_timewarrior_entry(new)
    subprocess.call(['timew', start_or_stop] + [ timewarrior_entry_text ] + [':yes'])

This will store (in Timewarriors database) the uuid of the Taskwarrior task that's being tracked. This gives us the advantage of being able to go back to Taskwarrior to get the correct information when reporting on the tasks that we worked. The downside to this is that using any of the built-in Timewarrior reports will only show the uuid of the task, and not the actual task information.

That's where the final piece of the puzzle comes in to play: a custom report for Timewarrior.

#!/usr/bin/env python3

## Note, this script lives in ~/.timewarrior/extensions
## as totals.py and needs to be executable

import os
import sys
import json
import datetime

from pprint import pprint
from tabulate import tabulate
from taskw import TaskWarrior

def formatSeconds(seconds):
  ''' Convert seconds: 3661
      To formatted:    1:01:01
  '''
  hours   = int(seconds / 3600)
  minutes = int(seconds % 3600) / 60
  seconds =     seconds % 60
  return '%4d:%02d:%02d' % (hours, minutes, seconds)


DATEFORMAT = '%Y%m%dT%H%M%SZ'
REPORT_INCREMENTS_MINUTES = int(os.environ['TIMEWARRIOR_INCREMENTS'] if 'TIMEWARRIOR_INCREMENTS' in os.environ else 15)
REPORT_INCREMENTS_SECONDS = REPORT_INCREMENTS_MINUTES * 60

# Extract the configuration settings.
header = 1
configuration = dict()
body = ''
for line in sys.stdin:
  if header:
    if line == '\n':
      header = 0
    else:
      fields = line.strip().split(': ', 2)
      if len(fields) == 2:
        configuration[fields[0]] = fields[1]
      else:
        configuration[fields[0]] = ''
  else:
    body += line

w = TaskWarrior()

# Sum the second tracked by tag.
totals = dict()
j = json.loads(body)

for object in j:
  start = datetime.datetime.strptime(object['start'], DATEFORMAT)

  if 'end' in object:
    end = datetime.datetime.strptime(object['end'], DATEFORMAT)
  else:
    end = datetime.datetime.utcnow()

  tracked = end - start

  groupDate = start.strftime("%Y-%m-%d")

  if groupDate not in totals:
      totals[groupDate] = dict()

  for tag in object['tags']:
    if tag in totals[groupDate]:
      totals[groupDate][tag] += tracked
    else:
      totals[groupDate][tag] = tracked

# Determine largest tag width.
max_width = 0
for tag in totals:
  if len(tag) > max_width:
    max_width = len(tag)

start = datetime.datetime.strptime(configuration['temp.report.start'], DATEFORMAT)

if 'temp.report.end' not in configuration:
    configuration['temp.report.end'] = datetime.datetime.utcnow().strftime(DATEFORMAT)

end   = datetime.datetime.strptime(configuration['temp.report.end'],   DATEFORMAT)

if max_width > 0:

  # Compose report header.
  print('\nTotal by Tag, for %s - %s\n' % (start, end))

  # Compose table rows.
  table_row = []

  for date in sorted(totals):
    sum_actual_seconds = 0
    sum_rounded_seconds = 0

    if len(table_row) > 0:
      table_row.append([ None, None, None])

    table_row.append([ date, None, None])

    for tag in sorted(totals[date]):
      taskName = tag
      id = None
      uuid = ""

      if tag.startswith('uuid:'):
        uuid = tag.replace('uuid:', '')
        id, task = w.get_task(uuid=uuid)

        if id is not None:
          task['tags'].append(str(id))

        taskName = "%s (%s)" % (task['description'], u', '.join(task['tags']))
        taskActive = None
        if 'annotations' in task:
            for i in reversed(task['annotations']):
                if i['description'] == 'Stopped task':
                    taskActive = False
                    break
                if i['description'] == 'Started task':
                    taskActive = True
                    break

        if taskActive:
          taskName = "> " + taskName

      taskSecs = totals[date][tag].seconds
      roundingSecs = datetime.timedelta(0, REPORT_INCREMENTS_SECONDS - taskSecs % REPORT_INCREMENTS_SECONDS if taskSecs % REPORT_INCREMENTS_SECONDS > 0 else 0)

      formatted = formatSeconds(totals[date][tag].seconds)
      rounded = formatSeconds((totals[date][tag] + roundingSecs).seconds)

      sum_rounded_seconds += (totals[date][tag] + roundingSecs).seconds
      sum_actual_seconds += totals[date][tag].seconds

      table_row.append([ taskName, formatted, rounded, uuid, task['status'] ])

    roundingSecs = datetime.timedelta(0, REPORT_INCREMENTS_SECONDS - sum_actual_seconds % REPORT_INCREMENTS_SECONDS if sum_actual_seconds % REPORT_INCREMENTS_SECONDS > 0 else 0).seconds
    table_row.append([ 'Total', formatSeconds(sum_actual_seconds), formatSeconds(sum_rounded_seconds) ])

  print(tabulate(table_row, headers=[ 'Tag', 'Time', 'Tracked', 'TW UUID', 'Status' ], tablefmt="orgtbl"))

else:
  print('No data in the range %s - %s' % (start, end))

Now, the vast, vast majority of this isn't needed if you're not dealing with rounding time to the next 15 minute increment (or 30 minute, as it's supported via Taskwarrior contexts), but this will give you a report (if you run timew totals) of the tasks that you've worked, by date (with the correct task title, as well as any tags), the actual time, the amount of time "tracked" (again, the next 15 minute increment bit), the Taskwarrior uuid (should you need to go look something up, and whether the task's status.

There is a minor performance issue that I'm (still) working through in that there doesn't seem to be an easy way to fetch all of the tasks for the provided uuids, so you fetch tasks individually - meaning that if you have time tracking for a large number of tasks, the report could take a while to complete.

I've needed this setup at multiple places now, and I find it pretty useful. I'm always looking to make the system better, so any tips or tricks would be appreciated!