I first tried to use UnboundDNS, but it seemed unreliable once modified for adblocking. I later discovered that dnsmasq does everything I expected from Unbound, but with the familiar configuration interface. It’s been battle-tested for adblocking, and so as a pre-requisite, enable and configure that.

Once you’re done, enable SSH and connect to your OPNsense box.

I used my phosphor user’s home directory to store my adblock files. Replace my username with yours where applicable

Steven Black maintains a nice hosts file that blocks a lot of things. We will download that and strip out the comments (dnsmasq requires this when loading extra hosts files).

1
2
3

mkdir adblock
cd adblock
curl -sSL "https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts" | grep 0.0.0.0 | grep -v '#' > hosts

Next go to your OPNsense Web GUi and navigate to Services -> Dnsmasq DNS -> Settings

In the Advanced section add the following, replacing my username with yours, or wherever you put your hosts file:

1

addn-hosts=/home/phosphor/adblock/hosts

You can add multiple hosts files this way if you wish. Finally click Save and then Apply Configuration.



Now you can test the adblock. You may need to reset your DNS cache on the clients you are testing. I like to use this site to test:

https://blockads.fivefilters.org



Hunting DNS queries to block

So dnsmasq can also log queries if you add log-queries to the advanced configuration section. Then, the opnsense dnsmasq logs will show queries.

Resources

• https://github.com/StevenBlack/hosts
• https://pi-hole.net/
• https://github.com/pi-hole/FTL
• https://github.com/collinbarrett/FilterLists

You will need:

• An active apple developer license
• A server in which to deploy sygnal

Push notifications in Matrix are configured by the client.

This is necessary due to the fact that that push is controlled by the device vendor. In other words, the iOS app tells the homeserver where the Sygnal server with the certificate corresponding to the App’s “App ID” is.

So you need to deploy a Sygnal server if you haven’t already, and load it with the certificate matching your the App ID you’ve compiled Riot as.

1. Modify the plist file for Riot, configuring the AppID you’ll use and the Push Gateway (Sygnal) URL. This URL is relative to Synapse, as synapse will be the one receiving the messages through other matrix users. It will use this URL to push the notification to your device.

   

2. Login to the apple developer portal at https://developer.apple.com

3. Go to the Certificates area, then go to “All”, and click the plus button

   

4. Choose the correct certificate for push, highlighted in green:

5. Go through the process of acquiring the certificate

   

6. Locate the certificate and keypair, and export it as p12.

   

7. Convert the p12 file to a pem file:

   1 2 3

   [keyvan@airframe ~]$ openssl pkcs12 -in ~/Desktop/Certificates.p12 -out apns.pem -nodes -clcerts Enter Import Password: MAC verified OK

8. Configure Sygnal to use this pem file:

   1 2 3 4

   [apps] pw.keyvan.riot.type = apns pw.keyvan.riot.platform = sandbox pw.keyvan.riot.certfile = /mnt/storage/apns.pem

Notes

You may want to do some maintenance on existing pushers in your Synapse database.

To list existing pusher entries:

1

select * from pushers;

To delete any existing pusher entries:

1

delete from pushers;

I had to do pagination in a react app today. Often times for things like this, it’s easier to write your own thing than to use a library. Here’s the Pager component I ended up with:

Pager.jsx

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

import React from 'react';

export const Pager = React.createClass({
  render: function() {
    const  {
      perPage,
      totalRows,
      pageNumber,
      lastPageNumber,
      query
    } = this.props;
    const checkDisabled = (change) => {
      let newPage = pageNumber+change;
      if ( newPage < 1 ) return true;
      if ( newPage > lastPageNumber ) return true;
      return false
    }
    const jumpToPage = (pageNo) => query({ perPage, showPageNumber: pageNo });
    const changePage = (change) => (event) => jumpToPage(pageNumber+change);
    return <span>
      <span className="total">
        total {totalRows} #{((pageNumber-1)*perPage) + 1}
      </span>
      <span className="pager">
        <button disabled={checkDisabled(-1)} onClick={changePage(-1)}>prev</button>
        <select value={pageNumber} onChange={({target:{value}})=>jumpToPage(value)}>
          { Array(lastPageNumber).fill().map((_,i)=><option key={i} value={i+1}>
            page: {i+1}/{lastPageNumber}
          </option>)}
        </select>
        <button disabled={checkDisabled(1)} onClick={changePage(1)}>next</button>
      </span>
    </span>;
  }
})

This pager component calls the query prop function (a redux action, say) in response to previous and next buttons, and direct page selection, with an object like so { perPage: 10, showPageNumber: 1 }

Naturally, this being react, how this is used and what query does is out of scope, unknown and irrelevant, but here’s a little screenshot of the UI in which this is being used, for kicks:

yield:

ingredients:

• 2 half chicken breasts
• 1/4 cup butter
• 6 cloves crushed garlic
• 2 cups seasoned dry bread crumbs
• 8 red potatoes

directions:

• Preheat oven to 400 degrees F
• prep chicken
  • In a small saucepan melt butter/margarine with garlic. Dip chicken pieces in butter/garlic sauce, letting extra drip off, then coat completely with bread crumbs.
  • Place coated chicken in a lightly greased baking dish. Combine any leftover butter/garlic sauce with bread crumbs and spoon mixture over chicken pieces.
• prep potatoes
  • Toss potatoes with oil, salt and pepper. Arrange, cut side down, on a large lipped cookie sheet or jellyroll pan.
• Put the potatoes on the lowest rack, and chicken around half-way. Bake for 45 minutes.

notes:

came out great! the oven configuration and timing seemed perfect.

the chicken was boring in the middle because of no marination, also i think the breading was pretty unnecessary / hid the taste of the garlic butter.

the potatoes were pretty amazing, especially how they became crispy on the bottoms

sources:

• http://allrecipes.com/recipe/8918/easy-garlic-chicken
• http://allrecipes.com/recipe/76954/roasted-new-red-potatoes
• http://www.sheknows.com/food-and-recipes/articles/978631/cook-several-dishes-with-one-oven

yield: 1/2 cup

ingredients:

• 5 tablespoons (65 g) sugar, such as organic cane sugar
• 1/2 cup (120 ml) sake
• 1 1/2 teaspoons pure cane syrup, such as Steen’s (optional)

directions:

1. Combine the ingredients in a very small saucepan, such as butter warmer/melter. Bring to boil over medium heat, give things a stir to ensure the sugar has dissolved.
2. Remove from the heat and set aside to cool. Taste and add the cane syrup for depth, if you like.

source: http://www.vietworldkitchen.com/blog/2013/09/homemade-mirin-recipe.html

yield: 1 cup

ingredients:

• 1/2 cups soy sauce
• 1/4 cups sugar
• 1/2 cups mirin
• 1/4 cups sake
• 1 garlic cloves crushed
• 1 slices fresh ginger peeled 1/8 inch thick
• 1 tablespoons water
• 1 tablespoons cornstarch

directions:

1. In a small saucepan, combine soy sauce, sugar, mirin, sake, garlic and gingerroot.
2. Cook over medium high heat 3 to 4 minutes.
3. In a small bowl, blend water and cornstarch.
4. Stir constarch mixture into soy sauce mixture.
5. Cook until thickened, stirring constantly.
6. Strain sauce.
7. Keep at room temperature for up to 24 hours.
8. Refrigerate.

source: http://www.food.com/recipe/yakitori-sauce-344705

In the past few years, more and more APIs have begun to follow the RFC5988 convention of using the Link header to provide URLs for the next page. We can do this too and it’s quite easy.

Here is a function I recently wrote to do this for the simple case of a big array:

1
2
3
4
5
6
7
8
9
10
11
12

function paginate(sourceList, page, perPage) {
  var totalCount = sourceList.length;
  var lastPage = Math.floor(totalCount / perPage);
  var sliceBegin = page*perPage;
  var sliceEnd = sliceBegin+perPage;
  var pageList = sourceList.slice(sliceBegin, sliceEnd);
  return {
    pageData: pageList,
    nextPage: page < lastPage ? page+1 : null,
    totalCount: totalCount
  }
}

To demonstrate the usage, imagine you have defined a function getMovies which provides a movieList array you wish to paginate.
You have an express route /movies which serves as the web API to your movie library.
You might create a paginated route like this:

1
2
3
4
5
6
7
8
9
10
11
12
13

app.get('/movies', function(req, res) {
  var pageNum = parseInt(req.query.page || 0);
  var perPage = parseInt(req.query.per_page || 50);
  getMovies(function(err, movieList) {
    if (err) throw err;
    var page = paginate(movieList, pageNum, perPage);
    if (page.nextPage) {
      res.set("Link", "/movies?page="+page.nextPage);
    }
    res.set("X-Total-Count", movieList.length);
    res.json(page.pageData);
  });
})

Note that in most cases, you would not be paginating from a big array. This was my first time paginating a fairly large set which was not from a database. In the case of database access, your function won’t be so general since it will depend on using the database API to create an efficient query by offset and limit.

A few days after this writing, a relevant item appeared on HackerNews discussing the use of an HTTP proxy for this purpose, which allows you to see TLS traffic in most circumstances, a shortcoming of my approach here with wireshark. Here is the link. The top comment recommends mitmproxy which looks like the better tool for the job in this case than wireshark! Still it is very good to learn so that you can intercept the traffic when lower level network functions are used directly, although this is becoming quite rare I think.

“Pokemon Go” is a mobile phone game in which little mofos spawn in various places in the real world (on a map) and you have to be within proximity to (a) discover them and (b) “catch” them by throwing a ball at them.

Finding these little mofos is a hassle because you don’t know where the optimal populations might be at any moment and/or you may be looking for a specific type of little mofo. If only you could see all the locations at once!

Someone created https://pokevision.com/ about a week prior to the writing of this article, however it is currently not working. It looks like this, showing you the exact locations and spawn timeouts of the little mofos anywhere:

I believe that pokevision was created by reverse engineering the communication between the mobile app and the backend game server, determining the API, and then using that artificially from the pokevision servers, caching the responses appropriately in a little-mofo-location-database.

If we are to do the same reverse engineering task, and this applies to any traffic on your mobile device (or any device with wifi, but restricted access, a mobile phone being just that), we need to setup a wifi hotspot that we control and monitor.

On macOS, this is very easy. A simple checkbox abstracts away the creation and configuration of a bridge in which your wifi becomes an infrastructure access point and NAT and DHCP are handled for you automatically:

Next up we open wireshark and select the bridge as our capture interface. This allows us to eavesdrop on the iPhone, assuming it is connected to our Mac via wifi.

Now the packets start flowing in:

Notice the protocol is TLSv2. It’s probably HTTP beneath that encryption layer. Wireshark lets us follow the connection, so the data stream is more readable than just straight packets:

In this view we can see that we have correctly identified traffic originating from the “Pokemon Go” app, but that a handshake is underway and in order to view anything else, we’d need to decrypt the encryption layer.

This all took some 20 minutes or so and got us an environment in which at least the ciphertext traffic was available to us, and with the right keys, plaintext-observable. I think that the pokevision team took this to the next level, using an android phone (probably rooted) to harvest the keys required to decrypt the traffic.

Because pokevision was created through reverse engineering, it probably won’t last. This explains why we are seeing this error despite the fact that the “Pokemon Go” app itself is currently operational. If I was niantic (owner of Pokemon Go), I would crack down on pokevision and add an in-app purchase in which the powers of pokevision were temporarily granted to a player.

Say you have configured VPN already and the interface is the default name, tun0.

Say you have Transmission installed and want to force it to work through the VPN only.

If installed on ubuntu, it should run as user debian-transmission which is a convenient handle by which to control its traffic.

Using iptables, we can, for any process whose owner is debian-transmission:

1. route packets destined to any machine on our LAN (192.168.1.0/24), thus allowing our HTTP client to work
2. after 1., drop packets only if they travel over any interface other than tun0

Thus:

1
2

iptables -A OUTPUT -m owner --uid-owner debian-transmission -d 192.168.1.0/24 -j ACCEPT 
iptables -A OUTPUT -m owner --uid-owner debian-transmission \! -o tun0 -j REJECT

I learned this from http://www.botcyb.org/2012/11/force-application-to-use-vpn-using.html

Persisting Across Reboots

There are a few ways to do this; here is the way I prefer it.

Save your rules off to a file:

sudo sh -c "iptables-save > /etc/iptables.rules"

Add up and/or down hooks to interfaces in /etc/network/interfaces, e.g.:

1
2
3
4

auto eth0
iface eth0 inet dhcp
  pre-up iptables-restore < /etc/iptables.rules
  post-down iptables-restore < /etc/iptables.downrules

I learned this from https://help.ubuntu.com/community/IptablesHowTo

So im working on something in node.js with justin (for fun and to
learn). it’s pretty neat i like it a lot for doing web stuff. im using
express.js for the backend.

I also prefer express and can’t seem to shake it as my go-to web “framework” – for me it is currently the quickest way to get a little web service going…

Ive been reading about different front ends, but I’m pretty confused.
Angular and React, do they do pretty much the same thing? The marketing
material makes them sound like very odd frameworks, but what the hell
does this mean? Can you use these without a backend like express (I
imagine you can if you dont want database access)? Or is it all UI stuff
in the front?

I know some of these do sever side rendering (which seems pretty cool)
but does this sort of just replace the templating engine on the express
side (I’m using swig which is just like jinja that django uses)?

Well it is indeed confusing.

So you probably know django is referred to as an “MVC” framework. Rails is also referred to as MVC.

Model view controller… Okay.
In Django/Rails the View is the part that renders a template (usually an HTML) using some templating language. The model brings your data forth for use in the View. The Controller serves this up in reaction/response to a web request.

The first thing to realize when speaking of web front-end frameworks (Which is what angular and react are, with react being slightly less and slightly more all at once [ill explain]) is this: in the above model of MVC, they are strictly working in the browser. This means they are 100% compatible with django/rails, whatever you’re using, because you are just including angular.js or react.js as a static javascript library and then following some convention thereafter to actually use the library to control your UI.

Sometimes people refer to Angular as “MVVM” or “MV*” because as people transitioned from traditional web frameworks like rails/django (request -> server-side template -> response) to SPA’s (Single Page App) they tried to fit this circle into the square peg of the way they were thinking before. It doesn’t really work, and is a shitty and confusing thing, but I only tell you in order to say this: when react came out it said “it is the V” in “MVC” or “MVVM”

This is still misleading though because react is not typically used like a server-side templating library, although it can sorta be (server-side rendering) but this is a pretty advanced usage and I actually still haven’t bothered trying it. It’s not the same as the server-side rendering of a template that we are used to in MVC/traditional frameworks, or maybe it is, who cares, you can read up on it and just know that with react you get it for free and it exists if/when you need to compute the UI state/DOM without the need for an actual DOM.

Just trying to figure out how the pieces of these thigns work together.
I mean I can pretty much use express just like I did django so this
makes 100% theoretical sense. How do these other things fit in?

Angular and React are just javascript libraries.

Angular involves writing html files in a way that binds it to data in a “controller” ( a javascript object that is like an instance of some UI component).

in angular you extend html itself by writing what are called directives. it’s pretty neat but i dont recommend it anymore now that react exists. (note I have not looked at angular 2 yet)

in react, you never write any html. this is useful because you are no longer coupling to the DOM or browser. in fact, the react package on npm knows nothing about HTML or the DOM… to “mount” a react component onto the DOM, you need the react-dom package.

this is what gives way to technology such as React Native, which is becoming highly competitive with ionic (angular-based) in the hybrid-mobile app dev framework arena.

react is a better investment, imo, but it gives you less out of the box…

i recommend this tutorial http://teropa.info/blog/2015/09/10/full-stack-redux-tutorial.html – you will see the functional programming style of react.

that’s the other way to think of them… in angular you are mutating all over the place and maintaining state in these controller instances (each one has a $scope object that has data dangling off it, to which the elements are “bound”). in react you have a very primitive way to manage state, but people tend not to use it, instead it is recommended (by the “flux pattern” of which “redux” is a popular implementation) to store state in a special object central to your app (or component)… known as the store.

the store serves to decouple app state from the view itself – only rendering via parameter passing, like a pure function (the react copmonent, then is a pure function).

mutation of the state occurs by means of actions that are dispatched to the store. the store has a “reducer” you wrote, which knows how to compute the next state based on the incoming action. that new state is then what gives way to the next “frame”, or rendering of the view for the new app state.