Terence Eden’s Blog 2014-03-11 Introducing Corkr at #NHTG14 https://shkspr.mobi/blog/2014/03/introducing-corkr-at-nhtg14/ What a crazy weekend! I made the last minute decision to attend Rewired State's "National Hack The Government 2014" hackathon. Rather than hack on any of the provided datasets, I wanted to work on an interesting way to present all the security flaws I had found in Government websites. I teamed up with Mark, Marcello, and Orlando - together we created "Corkr - Plugging the Government's Digital Holes" [Image: Corkr logo-fs8] We were looking for different and interesting ways to visualise the data. Interactive infographics, space-trees, leader boards, top-trumps - all classic staples of hackdays. In the end, we thought what would be the most visually impressive would be to present the sites on a map. That way we could show the scale of the problems - and let local authorities know where they needed to target. [Image: Corkr Main] It's hard to quantify just how bad the bugs are - is a single SQL Injection worse than two XSS flaws, for example - so we settled on just doing a raw count. Marcello and Orlando were able to convert the wpscan results files into JSON - which made doing the calculations a lot easier. Mark, our graphics / UI guy came up with a really strong design to help showcase just how bad some of the problems are. We decided against revealing the nature of the vulnerabilities and instead went for a traffic light system. [Image: Corkr Detail] Using OpenStreetMap and LeafletJS meant it was incredibly easy to create lush looking maps which were zoomable. Adding markers to specific locations was also really easy. Where In The Country Is A Website? Some websites have a defined geographic location - a tiny Parish Council's website can reasonably be centred on a map. The WHOIS data for *.gov.uk sites is not standardised. Very few of the sites spat out a Post Code when queried. Going to each website and looking for a "contact us" link was just too much work for a hackday - so we cheated! Google has a pretty good API for getting the geographic location from a search query. So I manually converted all the domain names into something Google could use. For example "10downingstreet.gov.uk" became "10 downing street" (used as an example - it's not a vulnerable site). Here's the PHP code: $location = $argv[1]; $json = file_get_contents('https://maps.googleapis.com/maps/api/geocode/json?address='.$location.',%20uk&sensor=false'); $obj = json_decode($json, true); echo $obj["results"][0]["geometry"]["location"]["lat"] . ", " . $obj["results"][0]["geometry"]["location"]["lng"]; Now, obviously that requires a lot of copying and pasting. Luckily, Linux makes it very easy to automatically copy the output of a script onto the clipboard: php -e locate.php West%20Byfleet | xclip -sel clip And then we created a master JSON file which tallied up the number of flaws, where the sites were, and what they were named. And then we did the same for the NHS sites! In the end, due to time constraints, we only managed to get around 200 .gov.uk sites and 150 .nhs.uk sites. And so, on very little sleep, we demo'd it... The Competition As ever with Rewired State, the standard of hacks was incredibly high. They ranged from the silly (dating MPs) to the serious (statistical comparisons of local areas). We presented towards the end, and were painfully aware of just how tough this was going to be. When you're up against a physicist using LIDAR calculated height data to infer shadow mapping and Pseudo-facial recognition using photoresistors, you know you're competing against the very best in the UK. I am delighted to say that we were one of the four winners! [Image: ] Rewired State@rewiredstate[Image: Twitter] And the winners are corkr, alertin time, dataforce and shadow mapping!❤️ 2💬 0🔁 018:43 - Sun 09 March 2014 [Image: ] Elliot Hughes@ElliotJH[Image: Twitter] #nhtg14 Corkr was one truly awesome hack, final winner❤️ 3💬 0🔁 018:11 - Sun 09 March 2014 [Image: ] Dan Palmer@danpalmer[Image: Twitter] Last but not least, Corkr, the government web security audit, was great. Really good to highlight security! #NHTG14❤️ 1💬 0🔁 018:12 - Sun 09 March 2014 [Image: ] Steve U@Steve_Upton[Image: Twitter] .@edent, @marcelloseri, @M6_D6 and @orliesaurus built a leaderboard of .gov vulnerabilities hacks.rewiredstate.org/events/nationa… pic.x.com/kry2iz0td8[Image: ] ❤️ 3💬 0🔁 016:58 - Sun 09 March 2014 [Image: ] Julia Higginbottom@gabysslave[Image: Twitter] And the final winner is Corkr #nhtg14❤️ 4💬 0🔁 018:10 - Sun 09 March 2014 Some Thoughts On The Day It was an absolutely amazing weekend. The food was good - and healthy - there was a wide range of people and some top notch government people there to talk with. It might have been nice to have a bit more of a formal "what are people working on" session at the start. With 80ish hackers it may have taken some time, but it would have helped people introduce themselves and recruit for teams. All in all, a minor concern. Prizes. I've written before about the corrupting influence of big money prizes. Luckily, the prizes on offer at #NHTG14 weren't jewel encrusted! About £30 - £40 worth of tech - plus some O’Reilly books. Just the right sort of level - not so extravagant that people would be gutted to lose, but expensive enough to make you go "Ooooh! What a lovely treat!". Of course, the real value is in the taking part. And that's not just hokum; it really is. [Image: ] Stef@stef[Image: Twitter] Such polished, thought through hacks. Hack-for-the-prize-money hackathons? Ha! I’d take a book and a cheer from my peers every time! #nhtg14❤️ 18💬 0🔁 820:14 - Sun 09 March 2014[Image: ] Terence Eden is on Mastodon@edent[Image: Twitter] Replying to @stef@stef damn straight. It's not about the prizes - it's about being around & taking inspiration from people doing *their* best.❤️ 2💬 2🔁 020:28 - Sun 09 March 2014 For me, I think the real prize is making the government take notice and fix their broken sites. I think that is slowly happening - but a bit more on that later. Congratulations to all the teams who took part - and many thanks to the judges for their impeccable taste. ------------------------------ 2012-11-26 Rewired State - UK Parliament 2012 https://shkspr.mobi/blog/2012/11/rewired-state-uk-parliament-2012/ This weekend, I went to Rewired State's Parliament hackday. I teamed up with amazing front end designer Max Bye and statistician par excellence John Sandall to create a data visualisation of Parliament's Demographics. Are the houses representative of the people in terms of gender diversity? Are the Labour Party younger than the Conservatives? Are the parties in the Lords particularly dissimilar? You can play with the hack or watch a video demonstration. Each bubble represents a political party The size of the bubble represents how many members they have The Y-Axis (Vertical) represents the average age of MPs / Lords The X-Axis (horizontal) represents how gender balanced the parties are (As you can tell, the hack was heavily inspired by Hans Rosling) Data A quick word about the data we used. The (beta) APIs had some reasonably good documentation - although the examples could have been better. It seemed to assume that a user was already intimately familiar with the (sometimes arcane) principles of Parliament. It also only spat out XML, so that needed to be converted to JSON. The main issue we had was with the quality of the data. Let's look at two examples. First, Linda Perham (picked solely because she's a mate of my mum!) { "FullTitle": "Linda Perham MP", "DateOfBirth": "1947-06-29T00:00:00", "DateOfDeath": { "-xsi:nil": "true", }, "Gender": "F", "Party": { "-Id": "15", "#text": "Labour" }, "House": "Commons", "MemberFrom": "Ilford North", "HouseStartDate": "1997-05-01T00:00:00", "HouseEndDate": "2005-05-05T00:00:00", "CurrentStatus": { "-IsActive": "False", "StartDate": { "-xsi:nil": "true", } } }, That's pretty comprehensive. We can see when she joined, left, her age, that she's still alive, and who she represents. Now, let's take George Galloway who has had an... interesting... Parliamentary career. { "FullTitle": "Mr George Galloway MP", "DateOfBirth": "1954-08-16T00:00:00", "DateOfDeath": { "-xsi:nil": "true", }, "Gender": "M", "Party": { "-Id": "26", "#text": "Respect" }, "House": "Commons", "MemberFrom": "Bradford West", "HouseStartDate": "2012-03-30T00:00:00", "HouseEndDate": { "-xsi:nil": "true", }, "CurrentStatus": { "-Id": "0", "-IsActive": "True", "Name": "Current Member", "StartDate": "2007-10-31T00:00:00" } }, All we have is his current status. It doesn't mention his previous life as a Labour MP, nor does it mention that he was the Respect MP of Bethnal Green in 2005. For MPs who have subsequently gone to the House of Lords, the data is also unhelpful. Betty Boothroyd was a Labour MP (for two different constituencies), then became The Speaker of the House of Commons, then went to the House of Lords. This is all the information we have on her. { "FullTitle": "The Rt Hon. the Baroness Boothroyd OM", "DateOfBirth": "1929-10-08T00:00:00", "DateOfDeath": { "-xsi:nil": "true", }, "Gender": "F", "Party": { "-Id": "6", "#text": "Crossbench" }, "House": "Lords", "MemberFrom": "Life peer", "HouseStartDate": "2001-01-15T00:00:00", "HouseEndDate": { "-xsi:nil": "true", }, "CurrentStatus": { "-Id": "0", "-IsActive": "True", "Name": "Current Member", "StartDate": "2001-01-15T00:00:00" } } There's also a significant lack of historical data. There are some Lords & MPs in the dataset who were in Parliament in the 1940s - but only a few. It would be great to have a comprehensive record of, say, the last 100 years. There needs to be a better representation of when a member has "changed" - whether that's affiliation, leaving and then returning, being elevated, changing constituency, or even gender. (Although, as far as I'm aware, there have been no Trans MPs. Nor any MPs with non ASCII characters in their name.) The data represents a very monochromatic view of the world. For examining broad trends, it was sufficient for a hackday. We had tried scraping Wikipedia to get full details of every election, but that was a bit beyond us (over 1000 people for every election, plus by-elections, for the last 50 years.) What We Found I was particularly surprised by how little gender diversity there is. 50% of the population is female, yet the Labour Party have roughly 33% women MPs. Caroline Lucas is the sole (female) representative of the Green Party - which doesn't quite balance out the entirely male Bishops in the House of Lords. In our data, you can see the big jump after the 1997 election - where the number female MPs doubled. Labour are consistently older than the Tories. That was completely against my expectations. So, play with the hack and see what you notice. Thanks As well as my amazing team mates Max Bye and John Sandall, I must thank the team from Rewired State; they put on a storming hackathon. There was plenty of interesting data, a good mix of people, healthy food and drink (as well as the obligatory pizza). While it would have been lovely to hold the event in Parliament - I appreciate that a hoard of geeks turning up with a panoply of dodgy electronics may not have best pleased the Serjeant-at-Arms. So The Hub Westminster was a fine substitute. Special mention to Alex Blandford who was very helpful at explaining the data and helping us navigate through the peculiarities of the system. Finally, massive thanks to the Speaker for this fine certificate. [Image: rewired state 2012 certificate] ------------------------------ ␃␄