Pesky's Blog

After years of delays, the drielandentrein finally started operating at the end of June this year. However, despite all the planning and waiting, the train connection between the three countries is not without its hiccups. Earlier this week, Rover, the public transport traveller’s association, expressed their disappointment with the reliability of the drielandentrein. For starters, Arriva still has staff shortages which started during the pandemic and shortages in train equipment. In an interview with L1 News, Arriva responded by saying they are actively recruiting and training drivers and take immediate action to repair any issues on the trains. It would be interesting to understand the nature of these equipment failures and if they were also observed during the tests. Read more...

Grafana has the ability to use Amazon Athena as a data source allowing you to run SQL queries to visualize data. The Athena table data types are conveniently inherited in Grafana to be used in dashboard panels. If the data types in Athena are not exactly how you’d like them in Grafana you can still apply conversion functions. In this case the timestamp column in Athena is formatted as a string, and I do not have the ability to adjust the table in Athena (which is normally what you’d want to do). If your Athena table doesn’t have a column in a DATE, TIMESTAMP or TIME format you will not be able to natively use a panel that relies on a timestamp. You can parse the timestamp string using the parse_datetime() function that expects the string, and the format the string is in. This columns will now appear as a timestamp to Grafana. Read more...

The Dutch Railway, NS, runs a few long train routes in the Netherlands and at times these routes may seem a bit “illogical”. An example of this in the July 2024 edition of the NS Panel news brief is the route from Dordrecht to Venlo through Schiphol. This train starts in the south, travels north to Schiphol and heads south again to Venlo. Even with all the major stops listed on the platform departure screen, this route may still confuse some passengers, especially if other quicker train connections exist between the two cities exist. Read more...

Exporting the results of a query can be useful to import it into other tools for data preparation or visualisation particularly as a CSV file. sqlite3 CLI Using the SQLite3 CLI allows you to set the query result mode to CSV and output that result to a file. A number of different output formats, including custom separators can be set as well. Read more...

Brussel-Zuid to Paris Nord routes between the High-Speed line Eurostar uses and the classic line used by EuroCity. Increasing both the frequency and capacity of cross-border trains is always a good thing, I’ve talked about this with the Drielandentrein between Belgium, the Netherlands, and Germany before. A train every hour that can be used by commuters between the cities of Luik, Maastricht, and Aachen. This was a route that previously required a transfer at Maastricht. Read more...

After almost a year Tram 12 is back at Amstelstation after being stopped by the renovation of the Berlagebrug. The over 90-year old bridge was due for maintenance after a few inspections and was closed for all traffic on 24 July 2023. This meant that Tram 12 which runs from Amsterdam Centraal Station to Amstelstation through Leidseplein and De Pijp had to stop one stop short at Amsteldijk during the renovations. The bridge re-opened for pedestrians, vehicles and ships, essentially all traffic except trams, on 10 November 2023. An issue with the rails meant further work needed to be done before trams were allowed on. Read more...

GitHub Markdown can render Mermaid diagrams including Mermaid Gantt charts. I discovered this from Simon Willison who shared Bryce Mecum’s post on using Mermaid Gantt diagrams to display traces. A Gantt diagram can be created and rendered within a code block tagged mermaid, a simple example copied from the docs can be seen below. gantt title A Gantt Diagram dateFormat YYYY-MM-DD section Section A task :a1, 2014-01-01, 30d Another task :after a1, 20d I previously used MarkWhen’s Meridiem online editor to create Gantt charts to visualise the service life on Dutch Sprinter trains, this required me to share screenshots of the chart unless I created a view-only copy. Using GitHub I can share a gist with the details of the chart, which others can then use or modify. The migration from MarkWhen to Mermaid isn’t perfect, but it’s a start. Read more...

I use python-frontmatter to read and parse the front matter in Markdown files used on the main site. The front matter is used to set page or post titles, along with categories and tags. The python-frontmatter library provides ways to load a file and parse the front matter making it available as a dict. However, there isn’t a documented way to write1 the front matter content to files if you’re starting from a dict.2 Read more...

Dynamically partitioning events on Amazon Data Firehouse is possible using the jq 1.6 engine or using a Lambda function for custom parsing. Using JQ expressions through the console to partition events when configuring a Firehouse stream is straight forward provided you know the JQ expression and the source event schema. I found it difficult translating this configuration into a CloudFormation template after initially setting up the stream by click through the console. Read more...

sqlite-diffable is a tool, built by Simon Willison, to load and dump SQLite databases to JSON files. It’s intended to be used through the CLI, however since May I’ve been using this tool as a callable Python module to build my main site after making changes to the code1. This has allowed me to combine the build commands into one Python file. The changes aren’t available in the upstream code2 so it’ll have to be pulled from my fork if you’re looking to use this. Read more...

I usually forget the syntax of defining enviornment variables on different platforms, so here’s a note for future me to look up.1 Bash/Zsh export VARIABLE_NAME=ABC123 Using export will set the environment variable within the current session, you can override the value by using export again on the same variable name. To apply this environment variable to all sessions set the variable within the shell’s startup script such as .bashrc or /etc/envrionment to be available by all users. Read more...

After recently discovering Untappd’s RSS feed, I set out to find if AniList had the same hidden feature so I could ingest activity logs to the homepage of pesky.moe. After much searching I found that this wasn’t the case, but AniList did have a an open GraphQL API! I was able to use this to get my user’s activity and pass them along when generating my homepage. This wasn’t enough. I have the urge to RSS all the things! I then set about creating an RSS feed from my users activity that I can fetch via the API. It wasn’t difficult given that I could reuse the code I had previously written both to fetch the activity and create RSS feeds from my other projects. Putting it all together I open-sourced the repository last weekend along with publishing the feed. Read more...

Within a repository’s settings tab under Security you can set an Actions secrets and variables. A secret is encrypted and created for use with sensitive data whereas a variable is displayed as plain text. I intended to use a repository variable as an environment variable for a Python script within one of my workflows and the GitHub documentation was not very useful in describing how to accomplish that. Thanks to mbaum0 on this discussion thread I learned that you can access variables with the vars context.1 Using vars I can then set the variable within the environment of the workflow to be used by the Python script. In the example below I have set USERNAME as an Action variable and then set it as an environment variable within the workflow. Read more...

Traveling by train across countries has been getting a lot easier, especially within central Europe where everything is a bit closer together.1 We’ve got the Thalys (now Eurostar) doing 12 trips a day from Amsterdam to Paris through Brussels, and the actual Eurostar doing 4 trips a day from Amsterdam to London, via Brussels and Lille. There are also other relatively good frequency and direct services to larger cities at high speed. Similarly, for slower travelers at longer distances, there is the European Sleeper operating night trains between Belgium, the Netherlands, Germany, and most recently the Czech Republic three times a week. The travel times here are longer than on a high-speed connection but you’re passing through many more cities along the way. The general idea is also that you’d have a good night’s rest throughout the journey. In this same category, OBB operates its nightjet service to many locations, like Zurich to Amsterdam, via Cologne and Dusseldorf, or from Munich to Rome. Read more...

Leaflet is a popular and feature-rich JavaScript library for displaying maps. One of these features includes creating and placing markers on a map with an icon, that could represent a dropped pin or something custom. I need to create icons for markers that also contain a number that could be different for each marker and adding text to Leaflet icons can be done in a few ways. Read more...

Cross account access to an S3 bucket is a well documented setup. Most guides will cover creating and applying a bucket policy to an S3 bucket and then creating a policy and role to access that bucket from another account. A user or service from that account can then assume that role, provided they’re allowed to by the roles trust relationship to acccess the S3 bucket via the CLI or API. Read more...

Since I’ve begun blogging semi-regularly I’ve learned that having visuals helps especially when it comes to talking about different locations and their relation to each other. It can be difficult to judge the space and distance between two streets in a city or two cities in a country without some familiarity with the location. A problem I come across frequently when reading transit-related articles and blogs. It’s for this reason I try to include maps in my transport-related posts from very early on. Read more...

Timetable or schedule graphs visualise railway traffic on a route during a set time. Typically when these graphs are shared online they look like they’ve been created using jTrainGraph although I find it very difficult to get started. I attempted to use both Google Sheets and LibreOffice Calc to some degree of success but only achieved what I wanted through by using R. This was made possible by a single StackOverflow question from 2011. Read more...

The MerwedeLingelijn stretches 49km from Dordrecht to Gorinchem across the Drechtsteden, Molenlanden, and Gorinchem (DMG) regions in Zuid Holland. The line is mostly single-track with passing opportunities at most stations. Like other public transport modes in the area, the trains on the MerwedeLingelijn are operated by Qbuzz. They began providing public transport in the DMG regions in 2018, taking over from Arriva. With that, Qbuzz also inherited 10 electric Stadler GTW trains to run services on the line. Eight of these trains are 3-car GTW 2/8’s and the other two are 2-car GTW 2/6’s. These trains were delivered and put in service between 2008 and 2011 while Arriva still ran the line. Read more...

As a long-time user of RSS (Really Simple Syndication) to consume media, I thought I’d write a short post about it. The first specification was released on 15 March 1999, so it did celebrate its 25th birthday yesterday after all. Subscribing to RSS feeds allows you to receive updates on new posts to a website, it’s used by news aggregators to get you articles from different sources, and most famously used to deliver the latest podcast episodes to your app of choice. 1 Read more...

Granting a user permission to access resources across AWS accounts is a common task, typically the account with the resources contains an IAM role with the appropriate policy defining the actions the user can perform. In addition to this a trust policy is created that specifies which principal can assume that role in order to perform the allowed actions. Sometimes an external ID is added to the trust policy which verifies the user wanting to assume that role. There are good reasons to use this, which I will refer to the AWS documentation for. Read more...

When examining passenger mobility, there are many angles from which to explore effective transportation systems: service reliability, cleanliness, comfort, cost, transit coverage, service frequency, and more. While this topic is extensive, I will specifically focus on the coverage of a public transit network within an area and the significant role that frequency plays in it. I will be using the public transit system in the region of Utrecht, which is operated by Qbuzz under the name U-OV, as a demonstration. The region includes nine municipalities and a population of over 700,000. A total of 46 routes are run by U-OV, four of which are light rail and the rest are bus. Read more...

AWS allows you to enable server-side encryption (SSE) for you data at rest in your SQS queue. Disabling this option also has an effect on your encryption in transit as well. From the SQS documentation: All requests to queues with SSE enabled must use HTTPS and Signature Version 4. In other words disabling SSE also means you can now communicate to the SQS without TLS. I wrote a Python script to send a message to a queue using the HTTP API and botocore without using the higher level abstractions of boto3. Note the endpoint used uses http and not https. Read more...

This week I open sourced a Terraform project I’ve been using for the past few months. This solution allows the user to schedule the start or stop of EC2 instances in a single AWS account. This schedule is defined through Terraform and created EventBridge Schedulers. This post will be a snapshot in time of how the solutions looks at the time of publishing. An up to date and concise description of the solution can be found on its GitHub page. Read more...

The Venlo train station has a great regional express connection to North Rhine-Westphalia (NRW), Germany every hour operated by Eurobahn from 5 am to 10 pm. The station also has stoptrein services to Nijmegen and Roermond along the Maaslijn, where they go twice an hour in both directions operated by Arriva. Lastly, it has an NS Intercity service to Dordrecht via Schiphol Airport, which is also twice an hour. As good as the timetable is, the services that connect you to and from the German train could be better. Read more...

On my Linux machine I have sometimes shared by screen as a virtual device so it can be used in other programs as a webcam. A similar FFmpeg can also just be used to capture video to the file as well. Capture as a video device The v4l2loopback module allows you to create a virtual video devices under /dev/. Read more...

This is the third installment about Sprinters in the Netherlands. In the first post we’ve taken a look at the emergence of more regional, sprinter-like services in the 1970s as the Dutch moved towards suburbanisation. This required a new trainset designed to provide stop-train services, which was called the Stadsgewestelijk Materieel (SGM) and was first purchased in 1972. After three decades of service, between 2003 and 2006, the fleet of 90 SGMs was renovated, extending their lifespan to 2021. We also touched on the failure of the Stoptreinmaterieel ‘90 (SM ‘90), which was supposed to be the successor to the SGM. In 2005, the plans for the Sprinter Light Train (SLT) were developed based on the proven Baureihe 425, following the failure of the SM ‘90. A total of 131 sets were ordered, containing four and six-part lengths, and delivery began in 2008. Read more...

This Christmas my wife and I booked a semi-spontaneous trip to Berlin, Germany for a few days. Our trip included visiting the typical Christmas markets, restaurants, East Side Gallery, and Museum Island. Most of our traveling around the city was done through the public tram network, with the occasional use of the U or S-Bahn. A line from Vladimir Nabokov’s 1925 short story, A Guide to Berlin, comes to mind. Read more...

You can track changes to a tag through AWS CloudTrail, AWS Config, or Amazon CloudWatch Events, these methods have already been documented but they’re too slow to respond to changes, too expensive to run, not as extensible out-of-the-box, or outdated. I haven’t seen much coverage on doing this with Amazon EventBridge, which has many integration options, is low-latency, and is fairly low cost (and in this case free). There is a page in the documentation titled Monitor tag changes with serverless workflows and Amazon EventBridge that covers just that, I’d recommend starting there. Read more...

To show related content on this blog I use Hugo’s in-built functionality, which works surprisingly well with no setup. I did, however, want to test out creating text embeddings from my posts and rank them by similarity. Before you continue reading, the usual disclaimer: Do not take the information here as good or best practice. The purpose of this entry is to post my learnings in somewhat real-time. I will use Amazon Titan Embeddings G1 - Text available through Amazon Bedrock and SQLite to store the results. The full code can be found towards the end of this post. Read more...

Amsterdam is a busy city with residents, businesses and visitors often talked about for its good urban planning, offering a walkable city that is still accessible by car, train, metro, tram, and bus. To those who have lived in or traveled around Amsterdam you will have come across the metro and their blue cubes with an “M” and red R-NET band at the bottom. If you’ve taken the time to travel on the Amsterdam Metro, does it seem like a typical metro? Read more...

It has been 87 days since I enabled the ability to show related content at the bottom of each post on this blog, aptly titled “Related ramblings”. This uses Hugo’s built-in Related Content functionality. If you use a pre-made theme you may never directly work with this feature. I wanted to highlight how easy it was to use and how impressed and I am with the results. To enable my theme to use this features, I simply had to create a new partial that could be used in my post template. The following snippet uses .Related which takes the current page and finds every related page according to the related configuration. The first 5 are picked and rendered onto the page by referencing this partial. To view the code you can refer to the commit on the theme. Read more...

Between 1990 and 1994 eighty-one Class 1700 locomotives were built by Alstom. These were an iteration of the Class 1600 that came before, based on the French BB 7200. Over the years they were used in a push-pull configuration for DD-AR coaches, ICRm coaches, and others. By the 2010s most of the locomotives were no longer used and in 2019 the 1700 locomotives were only used to pull the IC Berlin train from Amsterdam to Bad Bentheim by NS. Designed to only support 1500V DC, the locomotive is then swapped for a German one which then pulls the passenger carriages to Berlin. Read more...

Well, it’s time to revisit the topic of leaf fall. About a month has passed since my last post on the subject of adhesion and leaves on the track, and it is time for a quick update. As of 8th November, ProRail has deemed it unsafe to operate trains between Utrecht Centraal and Utrecht Maliebaan, the home of the railway museum. The culprit? Leaves on the track causing the train to go incognito. Read more...

It was a rainy Saturday afternoon when I was at a cafe with my wife scrolling through Hacker News when I came across Arne’s blog on why you should write your own static site generator (SSG). I had recently gone down that road myself, at least partially, so seeing someone’s views on the same topic intrigued me. I’ve been meaning to gather my thoughts around the subject, and what better time than now as somewhat of a response to Arne’s blog? Read more...

There have been a number of occasions where I needed to insert JSON objects into an SQLite database, the sqlite-utils Python library and CLI tool handled the task every time. I will be showcasing some of the JSON-inserting capabilities via CLI below. Inserting a single JSON object Here the object is stored in data.json, sqlite-utils takes in the insert command, the name of the database, the name of the table, the filename, and lastly, I also specify the column to use as the primary key. Read more...

To plot rail stations on a map for past blogs1 and toots23 I needed to fetch their longitudinal and latitudinal coordinates. This process can be described as geocoding, getting a coordinates from name or address. There are many powerful libraries out there that have the capability to do this, but I wanted something quick and simple to get started, this is where Nominatim comes in. In their own words: Nominatim (from the Latin, ‘by name’) is a tool to search OSM data by name and address and to generate synthetic addresses of OSM points (reverse geocoding). It has also limited capability to search features by their type (pubs, hotels, churches, etc). Read more...

It’s that time of year again, pumpkin spice lattes are in at Starbucks, sweaters are making their way out of storage and leaves are falling off trees. That last point concerns this thread more than the others. Damp leaves and train tracks are not the best match. Wheel-rail contact and adhesion The train wheel and rail meet on a very small surface area, called the contact patch. That small area must have enough friction to start and stop the train. The term adhesion is used to refer to the coefficient of friction, or grip, between the rail and the wheel1. Adhesion is measured in Mu (μ), which is a decimal number between 0.0 and 1.0 (or more than one if the force of friction is greater than the normal force). The lower this value the more slippery the contact patch is meaning there’s less friction. Typically more grip is required to start a train than the stop it.2 Read more...

What I like about Sass and other CSS preprocessors, despite my limited usage of them, is the ability to nest CSS elements to assign style rules to children. Earlier this year the first version of the CSS Nesting Module specification was published by W3C. This brings the feature of CSS nesting to the browser! According to CanIUse, nesting is already supported by all major browsers as of this year. Here’s a basic example, I have an article with a <h1> and <h2> header where I’d like all <h2> headers within the article to be blue. Read more...

I host all my images outside of Hugo to prevent any large files from residing in my git repository. This has led me to serve high resolution images in most of my posts which aren’t ideal for the end user. I recently learned about Hugos ability to get remote images, aptly titled resources.GetRemote, which then allows me to apply Hugo’s image rendering capabilities. This video by Eric Murphy covers some compressing and resizing methods within Hugo that I’ve applied to this blog. A detailed list of all the processing options can be found in the Hugo documentation. Read more...

Markdown is a neat and simple way to format text using a basic editor. You’ll have come across it on various different forums, instant messaging apps, and blogging platforms among other places where text fields are used. It was created by John Gruber in 2004 and had then been adopted across different platforms but lacked a standardised implementation. To address these inconsistencies, Jeff Atwood, John MacFarlane, and others have formed a group in an effort to standardise. Since 2014, text/markdown is now a recognized MIME type, and CommonMark was released which is the specification and test suite we need! Read more...

Update 2023-12-19: Got an update from the issue I raised that the AWS Backup Access Policy and IAM role issue has been resolved in the Terraform AWS Provider version v5.30.0 via this Pull Request thanks to @nam054 and @johnsonaj. They delay has now been added as part of the provider itself and I’ve confirmed it works! You can disregard the rest of this post or continue reading if you’re interested. I recently came across an InvalidParameterValueException when trying to add a newly created AWS IAM role as a principle within an AWS Backup access policy in Terraform. It worked after applying the Terraform module a second time. After multiple repeated trials I found the module always failed on the first attempt but succeeded on the second. It seemed odd and after an embarrassingly long time searching online, I came across a pattern in the reported errors in the issues on the AWS Terraform Provider repository. These included MalformedPolicyDocument, InvalidPolicy, InvalidParameterValue among others, all related to referencing recently created IAM resources. Read more...

The year is 1992, Sir Mix-a-lot’s Baby got Back is on the pop charts, and the Dutch railway (NS) is dominated by the Mat ‘64 trains, with a fleet of 246 units. These trainsets were introduced in 1961, making them 31 years old by 1992. Also during this time, the original Sprinter, the Stadsgewestelijk Materieel (SGM) is only 17 years old and approaching the middle of its expected lifespan. Read more...

The following post outlines how to use ffmpeg to take a number of images and create a video with each image present for a set duration with cross fades. The images are centered in the video with a black background. There are some instances when a previous image may be visible if it’s too large, it’s been a while since I tested this thoroughly so it could use some tweaking. Read more...

Sprinters in the Netherlands are easily recognisable by their blue and white livery. These trains are specifically designed to accelerate and decelerate quickly, making frequent stops along their route. In contrast, Intercity trains primarily serve large stations with fewer intermediary stops. During the 1960s and 1970s, the Netherlands went through a wave of suburbanisation, prompting the government to start the construction of planned cities such as Nieuwegein.1 It was during this time that the Dutch national railway company, NS, released “Spoor naar ‘75” a plan outlining the future of public transport in the country. The NS emphasized the importance of aligning city development with existing rail infrastructure and implementing integrated fare systems across different transport providers.23 Read more...

Disclaimer: Do not take the information here as a good or best practice. The purpose of this site is to post my learnings in somewhat real-time. AWS IAM Identity Center (previously and more commonly known as AWS SSO) allows you to control access to your AWS accounts through centrally managed identities. You can choose to manage these identities through IAM Identity Center, or through external Identity Providers (IdPs) such as Okta, Azure AD, and so on. AWS already has good documentation for all these sources. I have my identities on Keycloak which was a little bit difficult to set up for the first time, hence this post. Read more...

The contents of this post were originally toots on Mastodon, you can find the original thread there. @PeskyPotato 24 June 2023 19:34 Recently, I had the opportunity to ride the Elizabeth Line, a commuter rail and rapid transport system, much like the Paris RER, connecting greater London to its core. After starting construction in 2009 it was officially inaugurated on 17 May 2022, the line goes from Reading and Heathrow in the west to Shenfield and Abbey Wood in the east. As part of the construction, 10 new stations were built along with refurbishing some of the existing stations along the route making them more accessible. Read more...

European Sleeper is a Dutch startup created in 2021 with plans to run European night trains. Formed by Elmer van Buuren and Chris Engelsman, the train operator ran their first passenger service on 25th May 2023, from Berlin to Brussels, through Amsterdam, Rotterdam and Antwerp. From 2024, European Sleeper also plans on expanding past Berlin to Dresden and Prague. Since earlier 2022, I’ve been closely following the progress made by European Sleeper, anticipating their first ride. Although not a passenger myself yet, I was thrilled that this new train connection allows holidaymakers more sustainable, time efficient and dare I say entertaining travel opportunities. The Man in Seat 61 highlights some interesting travel connections that could be made using through the European Sleeper. Read more...

The contents of this post were originally toots on Mastodon, you can find the original thread there. @PeskyPotato 14 May 2023 11:56 This weekend we have the series 7000 and 9000 manufactured by AnsaldoBreda for the #Madrid #Metro 🇪🇸, in 2002 and 2006 respectively. They're designed for the wide guage lines and come in single and dual-voltage variants as the metro system use both 600V DC and 1500V DC through rigid overhead catenary. Read more...

The contents of this post were originally toots on Mastodon, you can find the original thread there. @PeskyPotato 6 May 2023 15:37 Not all Dutch trains are yellow! This is the Protos, an EMU built by Fahrzeugtechnik Dessau (FTD). Only 5 of these trains were ever put in service and they all run on the Valleilijn 🐓between Amersfoort and Ede-Wageningen operated by Connexxion since 2007. #train Read more...