Fundamentals principles of better security

Defence in depth

This is the daddy of all security advice. The idea is to not rely on any one particular technology, process or approach to security. By building up layers of defence you can tolerate the eventual compromise of one of the layers. For example, don't just run AV software and hope for the best, harden your system and keep it up to date. Then use technologies such as whitelisting, sandboxing and URL blacklisting to further reduce risks. Don't just rely on a perimeter firewall or NAT, firewall between networks and use IDSs.

Security is a process 

Similarly to the defence in depth advice, don't ever expect to be completely secure. No product or solution will ever make you 100% hacker proof, regardless of what the marketing material states. Security is a process of identifying vulnerabilities, prioritizing and mitigating risks, making things a little bit better each time. 

Security through obscurity

Sticking with obvious advice, don't rely on security through obscurity. This means relying on the secrecy of a design, implementation or process for security. For example, rather than relying on some obscure proprietary encoding to protect data, you should use well known and tested cryptography. It shouldn't matter that someone knows your data is protected with 256bit AES. 

Crypto is hard


Speaking of cryptography, it is difficult. Very difficult. So unless your clever idea to protect data has just been rigorously tested by the crypto community, you probably shouldn't use it. Use a standard library instead, don't roll your own. You're going to get it wrong. Even implementing standard libraries can be tricky, so be careful.

KISS

The phrase Keep It Simple Stupid applies to most areas of life, including security and IT in general. Basically, complexity is the enemy of good security. Complex systems are difficult to understand, manage and maintain. A unnecessarily complex system will degrade over time which generally means holes and weaknesses developing. Bugs creep in and remain hidden. People become afraid of it, unwilling to investigate and fix issues.

Test, test, test

Speaking of bugs. Test the security of code during development by means of code review and static analysis etc. Test your applications before they go into production, eg through audit, fuzzing and standard QA testing. Test your infrastructure once it's in production by red-teaming and using external audits / pen testing.

Temporary is rarely 

This might not sound like security advice, but it is. The classic being the "any/any" firewall rule. Probably got put there during development or testing. It fixed the problem, and probably has a comment like "TODO - Work around to get customer working before deadline. Must investigate and fix properly - A N Other, March 2005". Although unavoidable, at least don't pretend that you'll get around to fixing it later. Just accept that it is going to be yet another permanent hack, and perhaps invest another 10 minutes to make it suck slightly less.

Weakest Link

Aside from all that dodgy hacked code and long forgotten cronjobs, your systems are in pretty good shape. However, security is like a chain. It is only as strong as the weakest link. Perhaps your database server is uber-hardened. But if you've got a five year old cronjob running somewhere on an unpatched six year old server that connects to your database will full admin privileges, that hardening is meaningless.

Principle of least privilege

So, why would that cronjob that simply dumps data need to run as an admin? It was probably created when the database only had one user - the admin. Applying the principle of least privilege means that people and systems should only be authorised to do what is required for them to successfully do their jobs. Another example is developers having root access on production systems. Is that really necessary? Or even sysadmins having full access to the development source code repositories. Following this principle helps guard against accidental and malicious abuse of power.

You cannot create data retrospectively

But you can throw it away. This is an operational point that extends beyond security. You can do some very interesting and insightful things with operational metrics, giving you new visibility into your system and therefore greater understanding. From a security perspective, an IDS alarming might coincide with a jump in open file descriptors on a system, or a sudden jump in memory used. Perhaps the number of lines per minute in an application log file jumps or drops. By collecting, graphing and reviewing this data you could lower the time it takes to understand an attack. There is more to life than just network traffic. Not using certain data? Archive or delete it after 12 months. Simple.

Secure at the source

As a general principle, you want to secure as soon as possible. If data is being generated on a system, you wouldn't wait until that data has been transferred half way around the world before attempting to secure it. This can go as far as never letting unencrypted sensitive data even reach a harddisk.

Security is a user problem

On the one hand you have security, on the other you have usability. No matter how clever a security technology is, if it needs to be driven by a human then it will need to be usable. When designing a system, try to work security into it from the start so that it doesn't become a bolt-on that gets in  the way of usability.

Security is everybody's responsibility

Another classic principle, security is the responsibility of everyone, from the receptionist to the CISO. Why? Well, weakest link for a start. The receptionist might not have root access on the production servers, but they are probably on the network and could be a pivot in further attacks. They would probably also make good sources of spear-phising emails to internal addresses. Educate people to take an interest in doing their best to maintain security. Reward them for reporting suspicious activity (e.g. a social engineering phone call).

Remote code execution == game over

Basically, if a box has been compromised to the extent that the perpetrator has run code on it, you have to assume you have lost control of that system. Rootkits are becoming more sophisticated by the month and you probably do not have the time to fully investigate exactly what happened to the system. Investigate it, understand what they succeeded at, what they failed at and perhaps even what their motivation was. Then rebuild the box and start from scratch. Also, don't rely on backups taken after a compromise.

You're going to get hacked

Almost finally. It's not a matter of if, but when. Work on the assumption that you are going to get hacked and look at how it's going to happen and how you can minimize the impact on the business. Make sure you have an incident response process.The idea is to stop an attack as soon as possible, reducing the impact whilst recording data for later forensic analysis (a hardened centralised syslog server might help).

Finally

This list isn't complete. I will add to it over time.

London Sky - Flickr pictures

A slight tangent from the usual rare posts, but I've just uploaded some London skyline pics to Flickr.

The rest can be found here: Flickr

M4sl0w's hierarchy of needs (for geeks)

What is Maslow's hierarchy of needs?

I suggest you look here or Google for it, but in summary it is a psychological theory to describe the levels of human needs and desires and how higher levels of needs can only be satisfied once all the levels below are satisfied.

Does this apply to geeks?

Well, even geeks have basic needs that must be met. I'll discuss them in turn below.

Caffeine

All geeks need caffeine in some form. There are a few crazy exceptions, but they're by far the minority. Popular sources of caffeine are coffee and soft drinks. Energy drinks are of course an excellent source of this essential nutrient and have helped many a geek through a late night coding frenzy.

Awesome workstation and shiny gadgets

So, you're getting your regular dose of free caffeine. This will no doubt be the first thing you do when you get to the office, if you haven't already picked a drink up on the way in. You sit down at you're desk and you're greeted with what you're going to be staring at for the next eight hours. Do you look at it and have flashbacks to the eighties or do you admire the technical marvel before you? As they say, "if you need something done, you may as well do it on something cool and expensive".

Fun and interesting colleagues

As a geek you probably dread social interaction. Or perhaps not. I think that cliché is somewhat dated. But you probably have to work with people around you. Perhaps even regularly. The goal here is to have fun at the same time, building up a sense of team and therefore a shared sense of purpose. That way you are more productive and get things done sooner leaving you with more time to sip your caffeine drink of choice. If you're lucky, you might even meet your colleagues after work in some form of social event.

Interesting work, a sense of control and doing things right.

Do it. Do it now. Do it right now. Do it right.

So, you're sipping your regular dose of caffeine, sitting back at your shiny workstation after a interesting and thought provoking conversation with a colleague. It is now time to get on with the day and do some work.

I think most people, and especially geeks, have a basic desire work hard, work efficiently and to do the right thing. As a geek, you are being paid not just to mash the keyboard from time to time, but to use your expertise to assess and prioritize what work needs to be done and how. A sense of control is very important here - if you and your colleagues know what needs to be done, know how it needs to be done, know that it can be done and that it is the right thing to do, but cannot do it for political reasons, then you're probably going to be somewhat demotivated (wow, that last 'sentence' was an abuse of the English language, but you get my point I hope).

It is the role of a good manager to align the goals and opinions of the geek to the business requirements.

And, if you're lucky, your work might even be particularly interesting. Everything is interesting if you think about it the right way :) What was the other one? There are no boring tasks, only boring people.

Play time and training

Now that you're being productive and the business is happy with your and your team's work, what else is there?

Geeks do what they do because they love it, not because it is a career path (by definition, otherwise they're just random people who happen to work in IT etc). It is a geek's natural instinct to experiment, play, create and break things. And to learn during the process - understand how things work, what makes things tick and where the boundaries lie.

So, what is the ultimate geek motivation? To play, experiment, test, break, fix, create and get paid for it. That is why Google has their 20% time. This means that you get to spend 20% of your time doing whatever you like, which can sometimes result in the birth of a new product or service. It gives geeks a chance to think "outside of the box" (Bingo!) and to try or learn new things.

Why bother? Isn't it a waste of time? Well, perhaps. But there is an asymmetry in the potential gains. Basically, there is a fixed loss - 20% of an employee's time. However, the potential gain has no boundaries. Perhaps someone comes up with something truly phenomenal which ends up being worth millions to the business. Or perhaps they create something useful that gets release to the community and makes the world a slightly better place.

And who knows, the company might even lower its staff turnover and save some money on recruitment.

Finally, a word on training. There are two types of training found during employment. Basic training required to do the job effectively and personal training which indirectly benefits the company but adds to the employee's sense of purpose and career progression. Both forms of training are motivating, with the latter being more so. Unfortunately, training is rarely a priority for companies and doesn't seem to exist much in the wild. Lucky for them geeks tend to take responsibility for their own training, but this does generally result in higher staff turnover - if someone has dedicated most of their free time and spare cash into training themselves, why wouldn't they move on to reap the rewards of a better income or more interesting work?

10 useful little bash tricks

We're going to have a quick look at a collection of ten little bash tricks that you may or may not know. Or possibly forget. Or perhaps know, but don't tend to use often and so this might be a good reminder.

Please note, this stuff was tested on Bash 4.1.5. You can find further details and examples for all these tricks in the Bash manpage.

1::Using previous command

This trick is very simple. If you run a command and then have to repeat it (for example using sudo)
then instead of typing the whole command again you can refer to it using !!.

user@host:~$ aptitude install blah
[100%] Reading package lists^C
user@host:~$ sudo !!
sudo aptitude install blah
[sudo] password for user:
...
user@host:~$

As you can see, it even prints out the new full command.

2::Using previous argument(s)

Similar to the first trick, bash will allow you to easily reference the last argument of the previous command.

In this first example we're going to touch a few files, then decide to remove the last one.

user@host:~$ touch a b c
user@host:~$ rm !$
rm c
user@host:~$ ls
a  b
user@host:~$

As you can see, !$ only expanded to c. If you only had one argument, it would exand to that one argument.

We can also reference all the previous arguments using !*

user@host:~$ touch a b c
user@host:~$ rm !*
rm a b c
user@host:~$ ls
user@host:~$

3::Process Substitution

Right, now its time to move on to something a little different.

We want to compare the files in two directories to see which are in common, missing or changed etc. It would make sense to use the diff utility to help. We are also going to use md5sum to get a fingerprint of the files so we can see if they have changed. All we have to do is look through each directory (using find), do an md5sum of each file and compare the two lists.

Now, diff usually expects to operate on two files. We could redirect our results into files and then diff those files, but there is a smarter way. By using the <(list) for we can run our finds and diff will treat the results files.

diff <(cd boot-a; find . -exec md5sum {} \;) <(cd boot-b; find . -exec md5sum {} \;)

< 9c830b456ed37e0c0d63f2528fb43de5  ./initrd.img-2.6.35-23-generic
7,8d5
< f70fd0262a6f8e1e82028237e94657fc  ./config-2.6.32-25-generic
< c16c8e1705a54db8b96adce7de17710a  ./vmcoreinfo-2.6.32-25-generic
10d6
< b1f002028905e42f594c83603606ca0b  ./config-2.6.35-23-generic
12,16d7
< 668d54c704f22d85cb548f27a7e19d41  ./config-2.6.35-22-generic
< 882721477fcf705c02b60a4bb219b0c8  ./vmlinuz-2.6.32-25-generic
< 88472c27c3d18a832b5d85aea1edc541  ./abi-2.6.35-22-generic
< 496b4aed0005b07574e9dd3895089b23  ./abi-2.6.35-23-generic
< 6ee40238bdabb40a862c6f63262efa9d  ./System.map-2.6.32-25-generic
18d8
< 4e02de05c66cb0b322dc5c71f43882e9  ./System.map-2.6.35-23-generic
84c74
< ac9641014f0b460fc6eb5d2936bd869c  ./grub/menu.lst
---
> c5cce31e9e9eb884685137aac1ab8a8a  ./grub/menu.lst

As you can see, boot-a had quite a few files that boot-b was missing, and grub/menu.lst was different between the two directories.

4::Here document and redirect

Sometimes you need to put together a little config file, or perhaps a test HTML page. It is possible to do it quickly without having to resort to opening some sort of editor such as vi.

user@host:~$ cat << EOF > test.conf
> option1=blah
> option2=something else
> debug=true
> # end of config
> EOF
user@host:~$ cat test.conf
option1=blah
option2=something else
debug=true
# end of config
user@host:~$

What we did is specified a here document using << EOF. This keeps reading from the terminal until it encounters the string EOF. That is then redirected into the file, in this case test.conf.

5::List expansion when copying

This a very simple but extremely useful little trick that I'm sure you already know, but because of how often it is useful, I thought I'd put it here anyway.

Let us say you have a file you want to quickly backup. Instead of doing something like cp file file.backup we can use list expansion to shorten the command:

user@host:~$ ls
special.conf
user@host:~$ cp special.conf{,.backup}
user@host:~$ ls
special.conf  special.conf.backup
user@host:~$

6::Nested list expansion

Sticking with list expansion, it is worth remembering that you can have nested lists:

user@host:~$ touch {a{1..5},b{1,2,4},c{5..9}}
user@host:~$ ls
a1  a2  a3  a4  a5  b1  b2  b4  c5  c6  c7  c8  c9

7::Quick maths

Need to do a quick calculation? Already have a terminal open? Don't bother firing up a calculator, just do the calculation in bash:

user@host:~$ echo $((2**32))
4294967296
user@host:~$

See the Bash manpage for the supported operators.

8::Tilda expansion

Bash allows you to quickly reference the current $PWD and old one ($OLDPWD). This could be useful if you forgot to use pushd/popd:

user@host:~/a$ cd ../b
user@host:~/b$ ls ~+
inside_b
user@host:~/b$ ls ~-
inside_a
user@host:~/b$ echo $OLDPWD
/home/user/a
user@host:~/b$ 

So, ~+ expands to $PWD and ~- expands to $OLDPWD.

9::Setting default values

When you have a bash script that takes command line arguments you might want to set default values if no arguments are supplied by the user.

In the bash script below, we default arg1 to "a" if argument 1 is not supplied. We default arg2 to "b" and arg3 to "c". Of course, the order of the arguments matters.

#!/bin/bash

arg1=${1:-"a"}
arg2=${2:-"b"}
arg3=${3:-"c"}

echo $arg1 $arg2 $arg3

When we run the script without arguments, it uses the defaults. But when we give it arguments, it uses those values.

user@host:~/test$ bash test.sh
a b c
user@host:~/test$ bash test.sh x y z
x y z
user@host:~/test$

10::TCP and UDP

Finally, bash allows you to send TCP and UDP traffic directly using a special device of the format /dev/tcp/host/port (use udp instead of tcp if necessary).

Firstly, we set file descriptor 3 to use that device. We can do that with exec, as it suggests in the manual:

"Note that the exec builtin command can make redirections take effect in the current shell."

user@host:~$ exec 3<>/dev/tcp/localhost/80
user@host:~$ ls -la /proc/$$/fd/
total 0
dr-x------ 2 user user  0 2011-02-27 16:34 .
dr-xr-xr-x 7 user user  0 2011-02-27 16:34 ..
lr-x------ 1 user user 64 2011-02-27 16:34 0 -> /dev/pts/8
lrwx------ 1 user user 64 2011-02-27 16:34 1 -> /dev/pts/8
lrwx------ 1 user user 64 2011-02-27 16:34 2 -> /dev/pts/8
lrwx------ 1 user user 64 2011-02-27 16:34 255 -> /dev/pts/8
lrwx------ 1 user user 64 2011-02-27 16:34 3 -> socket:[156914]
user@host:~$

We quickly use $$ to see the open file descriptors for our process. As you can see, FD 3 has been opened for a socket connection, in this case a TCP connection to localhost on port 80.

We can now use redirect echo into FD 3. Note that we use the -e option so that bash interprets the escaped characters giving us new lines needed by the HTTP protocol. We then redirect into cat to read the output.

user@host:~$ echo -e "GET / HTTP/1.0\n\n" >&3
user@host:~$ cat <&3
HTTP/1.1 200 OK
Date: Sun, 27 Feb 2011 16:34:26 GMT
Server: Apache/2.2.16 (Ubuntu)
Last-Modified: Sun, 27 Feb 2011 16:25:27 GMT
ETag: "901-e-49d4602cddfad"
Accept-Ranges: bytes
Content-Length: 14
Vary: Accept-Encoding
Connection: close
Content-Type: text/html

<h1>Hai!</h1>
user@host:~$

Cool huh? Not a telnet, netcat or wget in sight...

A simple Monte Carlo simulation with R

I recently read How to Measure Anything: Finding the Value of Intangibles in Business by Douglas W. Hubbard. It's a fascinating and informative read on the problems and solutions of measuring "soft" variables typically found in business. Fluffy variables like productivity, quality, risk can be measured if you use the right techniques and work within the limitations of measurement and statistics.

There is an excellent example of using a Monte Carlo simulation (or method) to calculate the risk of leasing a new machine in a manufacturing process. You can find the example on pages 82 through to 86.

Given my hatred of spreadsheets and having recently started playing with R, I thought I would have a go at replicating the simulation using R.

This is what I wrote. Please note I'm still an R n00b so some things can be done better no doubt.

######################### Variables #######################

# Firstly set this to TRUE if we want to save our plot as a 
# PNG and if so, what file and dimensions
bDoPNG <- FALSE
sFile <- "htma.png"
iWidth <- 1024
iHeight <- 768

# The following values represent our 90% confidence interval 
# (CI) ranges for the various inputs to our simulation.

# We are 90% confident that the maintenance savins per unit 
# is between $10 and $20
vMaintenanceSavingsPerUnit <- c(10,20)

# We are 90% confident that the labour savings per unit 
# is between $-2 and $8
vLabourSavingsPerUnit <- c(-2,8)

# We are 90% confident that the raw material savings per unit 
# is between $3 and $9
vRawMaterialsSavingsPerUnit <- c(3,9)

# We are 90% confident that the production level per year 
# will be between 15K and 35K units
vProductionLevelPerYear <- c(15000,35000)

# The annual lease is $400K so we need to save this amount 
# just to break even for the investment
iAnnualLease <- 400000

# This is a quick cheat which basically means there are 
# 3.29 standard deviations in a 90% confidence interval
iStdDevCheat <- 3.29

# This is the number of simulations we are going to run
iNumberOfSims <- 100000

##################### Generate the basic data ###################

# A new data frame initiated to have iNumberOfSims rows in it
dData <- data.frame(seq(1,iNumberOfSims))

# We use the rnorm function to generate a distribution across 
# all the simulations for the maintenance savings. The mean is 
# literally just the mean of the range (e.g. (20-10)/2) and we 
# also give it the standard deviation of (20-10)/3.29.
dData$MainSavings <- rnorm(iNumberOfSims, 
mean(vMaintenanceSavingsPerUnit), 
diff(vMaintenanceSavingsPerUnit,1,1)/iStdDevCheat)

# Same again for the labour savings
dData$LabourSavings <- rnorm(iNumberOfSims, 
mean(vLabourSavingsPerUnit), 
diff(vLabourSavingsPerUnit,1,1)/iStdDevCheat)

# And the raw material savings
dData$RawMaterialsSavings <- rnorm(iNumberOfSims, 
mean(vRawMaterialsSavingsPerUnit), 
diff(vRawMaterialsSavingsPerUnit,1,1)/iStdDevCheat)

# And finally the production levels
dData$ProdLevel <- rnorm(iNumberOfSims, 
mean(vProductionLevelPerYear), 
diff(vProductionLevelPerYear,1,1)/iStdDevCheat)

# We can now create our total savings column based on the 
# inputs given. Because R is a vector language, the below 
# operation is applied to each row automatically.
dData$TotalSavings <- (dData$MainSavings + dData$LabourSavings +
dData$RawMaterialsSavings) * dData$ProdLevel

# Later on it will look better on the graphs if we deal
# with numbers in thousands so create a couple of shortcut variables
dData$TotalSavingsThousands <- dData$TotalSavings/1000
iAnnualLeaseThousands <- iAnnualLease/1000

# We now let R generate a histogram of our savings but without 
# actually plotting the results. We will end up with a series of 
# buckets (aka breaks) which will go on the X axis and the number 
# of simulations that fell within each bucket (on the Y axis)
hHist <- hist(dData$TotalSavingsThousands,plot=FALSE)

# We create a new data frame for the breaks and counts 
# excluding the last break
dHistData <- data.frame(
breaks=hHist$breaks[1:length(hHist$breaks)-1], 
count=hHist$counts)

# We can calculate the chance of the project making a loss as 
# the sum of counts where the breaks were less than 
# the annual lease (ie. $400K).
fPercentChanceOfLoss <- 100*sum(subset(dHistData,
breaks<iAnnualLeaseThousands,
select=count))/sum(dHistData$count)

# Calculate the median of the savings. That is 50% of
# the simulations had savings less than the median 
# and 50% had savings of more than the median.
fMedian <- median(dData$TotalSavingsThousands)

# We put that chance of loss in a sub title
sSubTitle <- sprintf("%02.2f%% chance of loss at $400K expense, 
median savings at $%02.0fK", fPercentChanceOfLoss, fMedian)

# Check whether we want to save our PNG
bDoPNG && is.null(png(sFile, width=iWidth, height=iHeight))

# Now draw the actual histogram, setting some labels but without 
# drawing the axis
hist(dData$TotalSavingsThousands,col="lightblue", main="Histogram of Savings",
xlab="Savings per Year ($000s in 100,000 increments)",
ylab="Senarios in Increment", axes=F)

# Add the sub title
mtext(sSubTitle)

# Draw the Y axis using default parameters
axis(2)

# Now draw the X axis explicitly setting values of the ticks/breaks
axis(1, at=hHist$breaks)

# That's it, turn off output if saving PNG
bDoPNG && is.null(dev.off())

And this is the pretty graph it produced. It should look similar to the one on page 86.

So yeah, go buy the book. Read it. Then have fun with R :)

Random spam

Literally... This one got through gmail's spam filters recently... URL in it is

"hxxp://www.oerstedfiltrate.info"

which seems to just be spamming about working from home. Had a quick look through the JavaScript, didn't see anything too scary. Interesting approach though.

Delivered-To: MYEMAILADDRESS
Received: by 10.204.52.201 with SMTP id j9cs78472bkg;
        Mon, 19 Jul 2010 03:02:26 -0700 (PDT)
Received: by 10.42.1.140 with SMTP id 12mr1676540icg.27.1279533745005;
        Mon, 19 Jul 2010 03:02:25 -0700 (PDT)
Return-Path: 
Received: from col0-omc2-s16.col0.hotmail.com (col0-omc2-s16.col0.hotmail.com [65.55.34.90])
        by mx.google.com with ESMTP id 1si10103284ibz.26.2010.07.19.03.02.24;
        Mon, 19 Jul 2010 03:02:25 -0700 (PDT)
Received-SPF: pass (google.com: domain of roger_to@hotmail.com designates 65.55.34.90 as permitted sender) client-ip=65.55.34.90;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of roger_to@hotmail.com designates 65.55.34.90 as permitted sender) smtp.mail=roger_to@hotmail.com
Received: from COL118-W13 ([65.55.34.71]) by col0-omc2-s16.col0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675);
  Mon, 19 Jul 2010 03:01:42 -0700
Message-ID: 
Return-Path: roger_to@hotmail.com
Content-Type: multipart/alternative;
 boundary="_075a8098-55f9-4a99-8412-f3c29427bff7_"
X-Originating-IP: [61.247.52.224]
From: ROGER JOEL TORIBIO 
To: 
Subject: Re: leg
Date: Mon, 19 Jul 2010 10:01:42 +0000
Importance: Normal
MIME-Version: 1.0
X-OriginalArrivalTime: 19 Jul 2010 10:01:42.0497 (UTC) FILETIME=[63661D10:01CB2729]

--_075a8098-55f9-4a99-8412-f3c29427bff7_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable


~]##=3D=3B_ma__ke=2C%$$~_[mo__ney{._@~=3D=3Bfas__t--]=3B~=3B$interne__t]~@=
=3D?~$

mghmqnc7 lqij0l44 c62u2h47oi9 c8x30ng s212x4cu y9mja8bfhh0 a5dy7n2mv4 5fycq=
2eeh6p sqz7rt eipy94g
nz30d86cgvx 6m7p5v7hd0 98v3nyk 5ohjr jf3mia7km42b y71nk f6owy5dzh w7s3h2 5l=
xnlp3g3m
tu36zwq9k 5g4fcglnu ypbrhwgnm9e ie3s2w yywbw 8hvfk76thv3 zj0c9sifp csft04h =
7j0b6fuh647m egw3vcw60w3
f537u3 o7pf805s cxtnm339vm f7or2a183r8 9c4q70rfp 5sisqgdr4ib demuk81dnob yn=
lhlw47 p22fyvd
c5f1uyii 1y6gs3t13w zbawrjdbco79 ju86sw i1mty2678n0 k8ac2xjr0 cecdlbounyjk =
pp3o 0jxerx 3p0552fepkq
6uyg0x2v2xdr l929yeslo8 4csmmc2 dbu5o ntcjv rom7mlpdm0xc cask yl30c oq8n26
dxdld8iw2fl asbgaqse280 r8hi y5y3e9 zik23m ned6 2a3jp7lgntcx sh3x2e0t
           =20
_________________________________________________________________
Hotmail: Trusted email with Microsoft=92s powerful SPAM protection.
https://signup.live.com/signup.aspx?id=3D60969=

--_075a8098-55f9-4a99-8412-f3c29427bff7_
Content-Type: text/html; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<style></style>
</head>
<body class=3D'hmmessage'>
~]##=3D=3B_ma__ke=2C%$$~_[mo__n=
ey{._@~=3D=3Bfas__t--]=3B~=3B$interne__t]~@=3D?~$

mghmqnc7 lqij0=
l44 c62u2h47oi9 c8x30ng s212x4cu y9mja8bfhh0 a5dy7n2mv4 5fycq2eeh6p sqz7rt =
eipy94g
nz30d86cgvx 6m7p5v7hd0 98v3nyk 5ohjr jf3mia7km42b y71nk f6owy5dz=
h w7s3h2 5lxnlp3g3m
tu36zwq9k 5g4fcglnu ypbrhwgnm9e ie3s2w yywbw 8hvfk76=
thv3 zj0c9sifp csft04h 7j0b6fuh647m egw3vcw60w3
f537u3 o7pf805s cxtnm339=
vm f7or2a183r8 9c4q70rfp 5sisqgdr4ib demuk81dnob ynlhlw47 p22fyvd
c5f1uy=
ii 1y6gs3t13w zbawrjdbco79 ju86sw i1mty2678n0 k8ac2xjr0 cecdlbounyjk pp3o 0=
jxerx 3p0552fepkq
6uyg0x2v2xdr l929yeslo8 4csmmc2 dbu5o ntcjv rom7mlpdm0=
xc cask yl30c oq8n26
dxdld8iw2fl asbgaqse280 r8hi y5y3e9 zik23m ned6 2a3=
jp7lgntcx sh3x2e0t
            
Hotmail: Trusted email with Microsoft=92s powerful =
SPAM protection.  target=3D'_new'>Sign up now.</body>
</html>=

--_075a8098-55f9-4a99-8412-f3c29427bff7_--

Happy birthday! Spam!!!!?

So, it was my birthday fairly recently, thank you, and while checking through my Google Mail spam folder I noticed several emails wishing me a "Happy Birthday" but which seemed to come from sites/organizations/addresses etc that I didn't recognize.

This got me thinking.

Is there any spam floating around which tries to use birthdays as a social engineering vector? Obviously you have a one in 365.25 chance of that person having a birthday on the day they receive the spam email (plus or minus a few days and assuming of course uniform distribution (which isn't the case I'm sure, but I think spammers could probably time things quite well) and that the spam makes it through whatever filters). Even with those odds working against the spammer, is the potential increased click-through/purchase etc worth it? In fact, if this Birthday distribution is anything to go by then I should imagine that September offers better than 1/365 odds.

Has anyone seen this sort of "vector", does it exist?

Just a thought.

kthnxbye

BTW, I think those emails I saw were things I had signed up for with my +spam@gmail.com address.
If so, hopefully I don't have to worry about it for another year or so...

Different crc16 and the problem with hashes

OK, so when I said "pretty slow", I think I actually meant that it is stupidly slow.

I'll get to exactly why in a second, but here is a slightly faster crc16 function (I "ported" it from some c code I found whilst googling, but unfortunately I cannot credit the original source as I cannot find it):

crc_table=(  0x0000 0x1189 0x2312 0x329b 0x4624 0x57ad 0x6536 0x74bf \
  0x8c48 0x9dc1 0xaf5a 0xbed3 0xca6c 0xdbe5 0xe97e 0xf8f7 \
  0x1081 0x0108 0x3393 0x221a 0x56a5 0x472c 0x75b7 0x643e \
  0x9cc9 0x8d40 0xbfdb 0xae52 0xdaed 0xcb64 0xf9ff 0xe876 \
  0x2102 0x308b 0x0210 0x1399 0x6726 0x76af 0x4434 0x55bd \
  0xad4a 0xbcc3 0x8e58 0x9fd1 0xeb6e 0xfae7 0xc87c 0xd9f5 \
  0x3183 0x200a 0x1291 0x0318 0x77a7 0x662e 0x54b5 0x453c \
  0xbdcb 0xac42 0x9ed9 0x8f50 0xfbef 0xea66 0xd8fd 0xc974 \
  0x4204 0x538d 0x6116 0x709f 0x0420 0x15a9 0x2732 0x36bb \
  0xce4c 0xdfc5 0xed5e 0xfcd7 0x8868 0x99e1 0xab7a 0xbaf3 \
  0x5285 0x430c 0x7197 0x601e 0x14a1 0x0528 0x37b3 0x263a \
  0xdecd 0xcf44 0xfddf 0xec56 0x98e9 0x8960 0xbbfb 0xaa72 \
  0x6306 0x728f 0x4014 0x519d 0x2522 0x34ab 0x0630 0x17b9 \
  0xef4e 0xfec7 0xcc5c 0xddd5 0xa96a 0xb8e3 0x8a78 0x9bf1 \
  0x7387 0x620e 0x5095 0x411c 0x35a3 0x242a 0x16b1 0x0738 \
  0xffcf 0xee46 0xdcdd 0xcd54 0xb9eb 0xa862 0x9af9 0x8b70 \
  0x8408 0x9581 0xa71a 0xb693 0xc22c 0xd3a5 0xe13e 0xf0b7 \
  0x0840 0x19c9 0x2b52 0x3adb 0x4e64 0x5fed 0x6d76 0x7cff \
  0x9489 0x8500 0xb79b 0xa612 0xd2ad 0xc324 0xf1bf 0xe036 \
  0x18c1 0x0948 0x3bd3 0x2a5a 0x5ee5 0x4f6c 0x7df7 0x6c7e \
  0xa50a 0xb483 0x8618 0x9791 0xe32e 0xf2a7 0xc03c 0xd1b5 \
  0x2942 0x38cb 0x0a50 0x1bd9 0x6f66 0x7eef 0x4c74 0x5dfd \
  0xb58b 0xa402 0x9699 0x8710 0xf3af 0xe226 0xd0bd 0xc134 \
  0x39c3 0x284a 0x1ad1 0x0b58 0x7fe7 0x6e6e 0x5cf5 0x4d7c \
  0xc60c 0xd785 0xe51e 0xf497 0x8028 0x91a1 0xa33a 0xb2b3 \
  0x4a44 0x5bcd 0x6956 0x78df 0x0c60 0x1de9 0x2f72 0x3efb \
  0xd68d 0xc704 0xf59f 0xe416 0x90a9 0x8120 0xb3bb 0xa232 \
  0x5ac5 0x4b4c 0x79d7 0x685e 0x1ce1 0x0d68 0x3ff3 0x2e7a \
  0xe70e 0xf687 0xc41c 0xd595 0xa12a 0xb0a3 0x8238 0x93b1 \
  0x6b46 0x7acf 0x4854 0x59dd 0x2d62 0x3ceb 0x0e70 0x1ff9 \
  0xf78f 0xe606 0xd49d 0xc514 0xb1ab 0xa022 0x92b9 0x8330 \
  0x7bc7 0x6a4e 0x58d5 0x495c 0x3de3 0x2c6a 0x1ef1 0x0f78 )

crc16()
{
    crc=0xFFFF
    for char in $(echo "$1" | fold -w 1); do
            byte=$(printf "%d" \'$char)
            (( crc= crc >> 8  ^ ${crc_table[($crc ^ $byte) & 0x00FF]} ))
    done
    echo $crc
}

In the above code the crc_table array would be a global variable.

Unfortunately, even though that crc16 function is somewhat faster, a BPA script of say 10 plugins each with 10 commands would still take just under 5 seconds to run. After thinking about it for five seconds, the reason is that that for each command (there are 10 x 10 = 100 of them) it needs to generate a crc16 about 5 times to store various bits of information. Then, once all the plugin data is loaded, there are relatively few actual reads of the hash which means that the script spends much much longer generating crc16 when inserting data, most of which is never going to be read and even if it is read it is only read a couple of times.

This is an extremely inefficient use of hashes, especially when written in something slow like bash.

So, what are the options?

• Redesign the BPA.sh script to not load all the plugins (which sort of defeats the purpose)
  
• Not use hashes in any form
  
• Use other fake hash made from running two arrays in parallel (one for keys and one for values)
  
• Use built-in associative array functionality of Bash 4.0 onwards
  

I think my cunning plan is to use a combination of the last two options. If the version of Bash is 4.0 or greater, then we can use the built-in functionality. If not, we degrade to the two array fake hashes.

I will be updating BPA.sh in the coming weeks then I guess...

Fun with bash: crc16, hashes and plugins

Today I released simple plugin architecture for bash scripts.

To do what it does, it uses three interesting features of bash:

• The source command to read in the plugins


• Bash arrays turned into hashes using a crc16 function written in bash


• Bash functions within functions to limit the scope of plugin commands

This post describes those three interesting features.

Bash Plugin Architecture

The project page is at: http://code.google.com/p/b-p-a/

It allows you to create plugins like:

user@host:~/bpa$ cat plugins/security/nmap.shp 
#!/bin/bash

registerPlugin "NMap" "nmap" "The mighty portscanner"

registerCommand 'SynScan' 'syn' 'TCP Syn scan of hosts'
registerCommand 'Ping' 'ping' 'Ping scan of hosts'

# Name of the plugin
NMap()
{
        # Name of the command.
        # Each command goes inside the Plugin() function
        SynScan()
        {
                # Code for this command
                hosts=$@
                nmap -sS -P0 $hosts
        }

        Ping()
        {
                hosts=$@
                nmap -v -sP $hosts 
        }
}

which can be used like

$ sudo security.sh --nmap --syn -- 192.168.0.1

For this to work, security.sh just needs to source the BPA.sh script and run a few commands (see the project page for details).

Sourcing plugins

This feature is probably the simplest, and part of the code of BPA.sh is below:

#
# Registers a new application.
#
# This must be the first thing called inside the application script.
#
# @param name
#   Name of the application.
#
# @param debug
#   (optional) If set to '1' debugging is enabled for the application. Defaults
#   to '0'.
#
# @param plugins
#   (optional) Sets the plugins directory for the application. Defaults to
#   './plugins' inside the directory of the application script.
#
# @return
#   Does not return a value
#
# Example:
# @code
#   new 'MyApp' 1 '/var/lib/plugins/myapp'
# 
new() {
        sAppName="$1"
        bDebug=${2:-"0"}
        sAppPluginsDir=${3:-"./plugins"}

        doDebug=$bDebug

        debug "New application $sAppName"
        for plugin in $(ls ${sAppPluginsDir}/*.shp); do
                debug "Sourcing $plugin"
                source "$plugin"
        done
}

Basically, when the application script security.sh sources the main script BPA.sh and calls the new function, looks inside a plugins directory for all files ending in .shp (shell plugin). For each file it finds it calls the source command.

Now, when you source a plugin, you run it's registerPlugin function and all of it's registerCommand functions which in turn uses the hash functions as described next.

Bash hashes

The registerPlugin function looks something like:

#
# Registers a plugin for the application.
#
# This should the first thing a plugin (.shp) script does.
#
# @param name
#   The name of the plugin.
#
# @param switch
#   The command-line switch used to access the plugin, using --switch.
#
# @param description
#   A description of the plugin. This is displayed when the --help switch
#   is used.
#
# @return
#   Does not return a value
#
# Example
# @code
#   registerPlugin 'MyPlugin' 'my' 'An example plugin'
#
registerPlugin()
{
        sPluginName=$1
        sPluginSwitch=$2
        sPluginDescription=$3

        debug "Registering $sPluginName"

        hash_set $sPluginName name "$sPluginName"
        hash_set $sPluginName description "$sPluginDescription"

        hash_set hPlugins "$sPluginSwitch" "$sPluginName"
}

The interesting thing in that function is the use of hash_set, which is:

#
# Creates hash with given name and sets key/value pair.
#
# @param name
#   Name of the hash.
#
# @param key
#   Hash key value.
#
# @param value
#   Hash value for key.
#
# @return
#   Does not return a value.
#
hash_set()
{
        name=$1
        key=$2
        value=$3
        lookup="${name}_keys"

        id=$(crc16 $key)
        eval $name\[\$id\]=\"\$value\"
        array_push $lookup $key
}

This function takes a name for the hash as its first argument, then a key followed by the value for that key. It then uses the crc16 function (shown below) to generate a numeric index for the actual bash array. It then uses the bash eval command to create a real array from the hash name that was supplied. Finally, it uses a simple array_push function to keep track of the keys in case we want to iterate over them.

Quite simple, and definitely overkill ;) It would be possible to just create two parallel arrays to track the key/value pairs, but this is more fun.

The crc16 function looks like this:

#
# Generates a crc16 value for the given string.
#
# @param string
#   String to be crc16'd.
#
# @return
#   Decimal crc16 value of string
#
crc16()
{
        # This could probably be more efficient...
        string=$1
        aString=($(echo $string | fold -w 1))
        cnt=${#aString[@]}
        crcPolyNom=0x8408
        crcPreset=0xFFFF
        crc=$crcPreset
        
        for ((x=0;x<$cnt;x++)); do
                char=${aString[$x]}
                byte=$(printf "%d" \'$char)
                for ((i=0;i<8;i++)); do
                        (( charLsb = $byte & 0x0001 ))
                        (( crcLsb = $crc & 0x0001 ))
                        (( doAdd = $charLsb ^ $crcLsb ))
        
                        (( byte = $byte >> 1 ))
                        (( crc = $crc >> 1))
        
                        if [ "$doAdd" -eq "1" ]; then
                                (( crc = $crc ^ $crcPolyNom ))
                        fi
                done
        done
        echo $crc
}

To get a value from the array, we just have to generate the crc16 for the key:

#
# Gets the value for key from the given hash.
#
# @param name
#   Name of the hash.
#
# @param key
#   The key value.
#
# @return
#   The value associated with the key
#
hash_get()
{
        name=$1
        key=$2

        id=$(crc16 $key)
        eval echo \$\{$name\[\$id\]\}
}

So yeah, this is pretty slow... but also pretty cool.

Functions within functions

You can declare a function within a function. For example:

NMap()
{
        # Name of the command.
        # Each command goes inside the Plugin() function
        SynScan()
        {
                # Code for this command
                hosts=$@
                nmap -sS -P0 $hosts
        }
}

In order to run the SynScan function, you must first run the NMap function. At first this might not seem very useful. But for a plugin system it is, as it allows multiple plugins to have the same commands, without any clashing.

BPA.sh calls the plugin function and command in the run function:

#
# Runs the actual application.
#
# @return
#   Does not return a value.
#
# Example:
# @code
#   run
#
run() {
        debug "Running $sAppName"
        parse $aAppArgs
        debug "Plugin: $sSelectedPlugin"
        debug "Command: $sSelectedCommand"
        $sSelectedPlugin
        $sSelectedCommand $aCmdArgs
}

So we could happily have another plugin, perhaps like:

OtherCoolPortScanner()
{
        # Name of the command.
        # Each command goes inside the Plugin() function
        SynScan()
        {
                # Code for this command
                hosts=$@
                othercoolportscanner -s $hosts
        }
}

...without worrying about mixups.

sp@m witt no spalling mistokes

Just received the following email:

from Adobe 
reply-to newsletter@download-adobe-pdf.com
to me@gmail.com
date 17 May 2010 16:40
subject Download New Adobe PDF Reader For Windows
mailed-by mmmail.co.uk
 
hide details 16:40 (1 hour ago)
 

PDF Reader for Windows - Compatible with All Windows Platforms

Get the new and improved 2010 version.
The ultimate PDF software pack to open, read and share PDF files online.

Download New PDF reader here:

http://www.download-adobe-pdf.com/

New Features in PDF Reader:

- Search, open & share PDF files online
- 50% faster startup then previous versions
- Search & save online Internet content
- NEW! Improved user interface

Also receive the award winning OfficeSuite absolutely free.

New version available :

http://www.download-adobe-pdf.com/

Since the 90's, PDF has become the standard file format for document exchange. - Adobe

Thank you,

Adobe Reader Support



Click here to unsubscribe 

Not only did this get through the usual spam filters Google so wonderfully provides, but it actually looks quite professional =/

A quick look at the whois output...

Domain name:             DOWNLOAD-ADOBE-PDF.COM
Name Server:             ns1.download-adobe-pdf.com 67.225.133.235
Name Server:             ns2.download-adobe-pdf.com 67.225.133.235
Creation Date:           2010.05.17

Status:                  DELEGATED

Registrant ID:           UALWS2Z-RU
Registrant Name:         Tommy Anderson
Registrant Organization: Tommy Anderson
Registrant Street1:      273 Parkway
Registrant City:         Miami
Registrant State:        FL
Registrant Postal Code:  33402
Registrant Country:      US

Administrative, Technical Contact
Contact ID:              UALWS2Z-RU
Contact Name:            Tommy Anderson
Contact Organization:    Tommy Anderson
Contact Street1:         273 Parkway
Contact City:            Miami
Contact State:           FL
Contact Postal Code:     33402
Contact Country:         US
Contact Phone:           +1 504 3442930
Contact E-mail:          tommy3849@hotmail.com

Registrar:               Regional Network Information Center, JSC dba RU-CENTER

Last updated on 2010.05.17 20:38:33 MSK/MSD

and suddenly it doesn't look so professional.

I haven't actually tried visiting the site yet, I don't have any systems I can happily pwn.

The unsubscribe link goes to:

http://www.mailingm.co.uk/3/unsubscribe.php?M=2723658&N=2435&=4&C=fbe0b28cf0f58710c5b4dea8a6e2ca68

A quick Google search:

Mailingm.co.uk  - Online Marketing Services Provider | Visit ...
This company specializes on marketing services through e-mail giving their clients the possibility of reaching a wider and international audience.
www.killerstartups.com › Site Reviews - Cached - Similar

In conclusion, this might just be some dodgy marketing for an Adobe reseller, but I don't think I'll take my chances.