Now, I just need to try posting new stuff a bit more often. Six years is too long to go between posts.
]]>For my tests, there are three groups of servers involved: my personal mail server (we'll call this the "PWB server"), my employer's mail servers (the "LT servers") and Google's mail servers (the "Gmail servers"). The PWB server's MTA is Postfix. The LT servers include Postfix relays and Kerio Connect mail servers. Mail sent out from the LT servers is first handled by Kerio Connect, then relayed to the outside world by Postfix.
I decided to limit my test to a single attachment - a TIFF file I picked out of convenience. This file is named eyes_color.tif and it is 196926 bytes in size.
I conducted several tests, but limited this analysis to a representative batch:
From all four tests, I extracted the base64-encoded attachment. The results from Test 1 and Test 2 matched, and decoding those gave back the original TIFF. The SHA1 hashes verified this. This correct base64 content is in good.base64. Both Test 3 and Test 4 included corrupted bytes - just a few each. The extracted base64 content was the correct size, but each had a few bytes replaced with non-ASCII characters. The corruption was different between the two and seemingly random. Test 3's extracted base64 content is in bad-1.base64, while Test 4's is in bad-2.base64. Running diffs between Test 3's attachment and the correct base64 content and between Test 4's attachment and the correct base64 content yielded the bad-1_v_good.patch and bad-2_v_good.patch, respectively. When viewed in Gmail's web interface, the attachments from Test 3 and Test 4 fail to show previews and, when downloaded, are not viewable in an image viewer. The attachments downloaded from the web interface do not match the original file sent in those emails, verified by SHA1 hashes.
These tests are representative of all of the tests I conducted. Gmail seems to corrupt attachments sent from some, but not all, servers. I do not know why, and I do not see a pattern in how the attachments are corrupted. When the same sending servers deliver mail to other servers, the attachments arrive in perfect condition. The corruption seems to be conditional upon the sending server, as I get consistent results with repeated tests from any given account. I have only used a few accounts on each server to conduct my tests, so it may be conditional upon the sending mail account, but this seems less probable. In each corrupted attachment, a different handful of seemingly random bytes have been replaced with non-ASCII characters. I know that this corruption affects multiple file types, but I do not know if all file types are affected.
With assistance from folks at Google, I have identified a probable source of the corruption in the network path between the affected sending servers and the Gmail servers. I do not yet know why other receiving servers are unaffected, but it may be a difference in error detection and correction behavior (like TCP checksum behavior) or a performance difference that affects the chances of corruption. If the affected network provider gets their problem fixed, I will conduct further testing.
Comments from this post were discarded during a website migration.
Referenced files have been removed.
]]>Today, Cardstore.com sent out an email to customers without using either of the above methods. This email contained thousands of Cardstore.com's customer email addresses in the email's "To:" field, meaning that every recipient of the email could see every other email address that the message was sent to.
Database breaches have become extremely common. LinkedIn and Last.fm are both recent examples of popular websites to suffer database breaches that exposed customer details. These breaches have been the result of hackers, but Cardstore.com cut out the middle-man and just sent out their customer list themselves. This kind of breach is unacceptable and should never have been allowed to happen. Great care should always be taken in the handling of customer information, and checks should be in place to make sure that errors like this are avoided.
Comments from this post were discarded during a website migration.
]]>On Wednesday, something happened that has never happened before: a coordinated effort by several of the most heavily-trafficked websites changed the course of political dialogue. Following Wednesday's blackout protests, SOPA has been pulled from consideration and PIPA is effectively dead in the water. Politicians were quick to react when overwhelmed by feedback from their constituents. This is a strong message that shows just how much power new media now wields. It took a unique circumstance to bring these major Internet players together behind a common cause, but now we know that it can happen.
As much as I would like to see the general public exercise greater everyday vigilance toward lawmaking, I do not think we will see much change there as a result of this event. What we might see is a stronger and more unified political voice from new media companies interested in protecting the greatest tool for information exchange ever created. I certainly hope that this will at least remind lawmakers that they should seek more input from major Internet organizations when crafting laws affecting the Internet. I do not doubt this event will impact legislative discourse in the future. It may prove difficult to trace the effects, but the world did change on the 18th of January, 2012, and I suspect for the better.
Comments from this post were discarded during a website migration.
]]>SOPA and PIPA are intended to give copyright holders the tools they need to bring down and block access to websites hosting copyright-infringing materials. While it is easy to see how blocking copyright-infringing websites would be desirable, concerns include that these tools may be too broad, that they may be abused, and that the burden on websites to avoid infringing or linking to infringers could be too great. Today's blackout is intended to raise awareness about these two bills. Visitors to Google will see a large black box over the Google logo, accompanied by a link to information placed on the homepage. Wikipedia has blocked access to most of its English-language pages. Reddit, imgur and others have completely gone dark, replacing their homepages with messages about these bills. Today's blackout is unprecedented in the history of the Internet.
Could Google be expected to find and remove all of the infringing websites among the over one trillion URLs it has indexed, and to continue monitoring each and every one of them for new infringement? Many infringers are going to try hard to avoid scrutiny, and there is a very large grey area where it might be hard to decide what constitutes infringement. What happens when they accidentally identify false positives? When they miss some infringement? These are the concerns that search engines face. What about sites like Facebook, Twitter and Wikipedia, which depend upon user-submitted content? They cannot possibly filter every single URL that passes through, and they could be deluged with takedown demands if they do not. The free flow of information and ideas that normally takes place on social networks would be stifled, and communities like Wikipedia that are driven by user contributions could be overburdened by the administrative demands. However legitimate websites might be forced to filter their content, we are facing a form of censorship never before seen on the Internet. In spite of whatever good intentions might be behind SOPA and PIPA, the burden they place on legitimate websites and the threat they present to the freedom of the Internet cannot be ignored.
Small businesses and individuals running websites would be most vulnerable to unintended consequences of SOPA and PIPA. Even small websites, blogs and forums could be forced to censor content or face being shut down. Funding for small websites that host user-generated content, whether in the form of comments and discussions, videos, articles or anything else, would be harder to come by when those websites could be held liable for infringing content. Operators of such websites would also face an increased risk of lawsuits, justified or not, which could prove too expensive to fight. Other tools used by small businesses could also be endangered. Mailing lists, code repositories, VPNs and more could pose liability concerns.
Copyright infringement is an expensive problem for American businesses. Content producers and publishers lose a great deal of money to piracy every year. Estimates on the actual losses vary greatly, but "many billions" is a good guess. Many websites that host the infringing material are outside of the United States, often in places that do not offer strong protection for intellectual property. This presents a challenge for American businesses, as it can be impossible to sue the infringers or their hosting providers, and the Internet as it is does not offer a way to shut down these sites. SOPA and PIPA are intended to answer this need by giving copyright holders a way to cut infringing sites off from the traffic that sustains them. Most of the opponents of SOPA and PIPA recognize that defending intellectual property is important - they often depend upon it themselves. The objection is over how these bills propose to cut infringing sites off.
Google and other opponents of these bills do have an alternative in mind: the OPEN Act. The OPEN Act aims to cut off infringing sites by stopping the flow of money to them. Status SOPA has been temporarily halted in the House, with discussion expected to resume next month. PIPA is expected to go before the Senate on the 24th of January.
Electronic Frontier Foundation articles:
Protest Letters:
Comments from this post were discarded during a website migration.
]]>What follows is a description of the process I used to get Fedora 15 running with the stock Fedora kernel on Rackspace Cloud. Rackspace Cloud, like Amazon AWS and many other VPS providers, uses the Xen hypervisor. Under a typical configuration, custom kernel and init images are used for each VPS, rather than images stored within the VPS. The kernel used with Rackspace's Fedora 14 image is a custom kernel built on Ubuntu. Thankfully, Rackspace does allow operators to use PV-GRUB to load other kernels. If you are considering trying this process on a production system, seek therapy. Start with a new server and migrate your services once you're done. If this doesn't go smoothly, you could be left with a server that will not boot and no recourse but to restore a backup.
The general process goes something like this:
The fist step is to set up a server loaded from Rackspace's Fedora 14 image to run as a Xen domU loaded by PV-GRUB.
cat >> /etc/modprobe.d/domu.conf << EOF
alias eth0 xennet
alias eth1 xennet
alias scsi_hostadapter xenblk
EOF
sed -i 's/sda/xvda/g' /etc/fstab
echo "hvc0" >> /etc/securettyThe next step is to install the Fedora stock kernel.
yum install kernelTake note of the kernel version. You'll need it in the next step. You can get the exact version string you need with this command:
rpm -q kernel --qf '%{version}-%{release}.%{arch}\n'Time to create the configuration file required by PV-GRUB. We'll create this as grub.conf and create a symlink at menu.lst, which is how the grub configuration is normally created on Fedora. In the following, replace "$KERNELVERSION" with the correct kernel version string.
mkdir -p /boot/grub
cat >> /boot/grub/grub.conf << EOF
#boot=/boot/grub/stage1
default=0
timeout=1
title Fedora ($KERNELVERSION)
root (hd0)
kernel /boot/vmlinuz-$KERNELVERSION ro console=hvc0 root=/dev/xvda1 SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=us
initrd /boot/initramfs-$KERNELVERSION.img
EOF
ln -s ./grub.conf /boot/grub/menu.lstSince we're going to use the stock Fedora kernel, we can set things up for yum to handle kernel updates and have the grub.conf file updated automatically. The commented-out boot directive in our grub.conf is part of this configuration. To finish this setup, we'll need to install grub and grubby and create a few more configuration items.
yum install grub grubby
cp /usr/share/grub/x86_64-redhat/stage1 /boot/grub/stage1
ln -s ../boot/grub/grub.conf /etc/grub.conf
cat >> /etc/sysconfig/grub << EOF
boot=/boot/grub/stage1
forcelba=0
EOF
cat >> /etc/sysconfig/kernel << EOF
# UPDATEDEFAULT specifies if new-kernel-pkg should make
# new kernels the default
UPDATEDEFAULT=yes
# DEFAULTKERNEL specifies the default kernel package type
DEFAULTKERNEL=kernel
EOFTest that grubby detects grub.
grubby --bootloader-probeAt this point, it's time to contact Rackspace Support to enable PV-GRUB. They will enable PV-GRUB and reboot your server. With a little luck, you server will start up with your stock kernel. Reconnect to your server and verify.
uname -aAt this point, you should consider creating an on-demand backup.
Now that you are running on the stock Fedora 14 kernel, you are ready to upgrade your server to Fedora 15 (and its kernel). The general Fedora guidance is here: https://fedoraproject.org/wiki/Upgrading_Fedora_using_yum Because we're dealing with a cloud server with PV-GRUB, there will be a few differences. You won't be switching to a text console or changing runlevels. You will not want to install the other Base packages. You cannot write a new MBR with grub-install. Really, our process is much simpler. To be safe, we'll use screen to launch our upgrade to mitigate the risk of a disconnect. (I recommend always using screen when using yum or doing anything else critical over SSH.)
yum install screen
screen -h 10000 -S yumThis screen command will increase the default history size and name the session for easy access. If you become disconnected during the upgrade, connect to the server again and run the following command to attach to your screen session:
screen -rx yumTime to perform the upgrade.
rpm --import https://fedoraproject.org/static/069C8460.txt
yum update yum
yum clean all
yum --releasever=15 --disableplugin=presto distro-sync
cd /etc/rc.d/init.d; for f in *; do /sbin/chkconfig $f resetpriorities; done
ln -sf /lib/systemd/system/multi-user.target /etc/systemd/system/default.targetMake sure that the new Fedora 15 kernel is configured and set as default in /boot/grub/grub.conf.
Reboot. A little more luck and your server will come back up running the new Fedora 15 kernel and userspace. You may want to create an on-demand backup at this point. You can reuse this image to create more Fedora 15 servers, but do not forget to contact Rackspace Support to enable PV-GRUB for each instance. If you created an on-demand backup after getting Fedora 14 running with PV-GRUB, you can now delete that image. Have I missed anything? Let me know.
Comments from this post were discarded during a website migration.
]]>Both the administrator and mentor roles in the Summer of Code are wonderful opportunities. Karsten lists the benefits and caveats of the administrator role in his post, and I think he sums them up perfectly, so I will not repeat them. There are some important points I would like to add about being an administrator with Fedora for the Summer of Code:
If you might like to serve as an administrator for Fedora's participation in Google's Summer of Code, please contact me right away.
Comments from this post were discarded during a website migration.
]]>In 2008, Google required the Fedora Project and JBoss.org teams to apply as a single organization to the Summer of Code, since both are Red Hat-sponsored open source projects. While that created some hurdles for us, since we are two very distinct communities with almost no overlap, we met the challenge head-on and created our "Red Hat Summer" group to coordinate our efforts and bring the teams together for our common goal.
"Give a man a fish, and you have fed him for a day. Teach a man to fish, and you have fed him for a lifetime."
As soon as Fedora and JBoss.org began working cooperatively, we also began to focus on making our efforts and resources more generic, such that we could take "Google Summer of Code" and replace "Google" and "Code" with any sponsor and any deliverable of interest. We began slowly working on an architecture that would allow us to support other seasonal development for students without a strict dependency on Google's program. We have greatly appreciated what Google has done with the Summer of Code program, and we are disappointed that we will not be participating this year. I have enjoyed my role as a mentor since 2005 and organization administrator since 2006, and my summer just won't be the same, but we are going to take the opportunity to quickly advance our non-GSoC summer coding initiatives.
We will still welcome students to work under our guidance this year, and we are hard at work to find sponsors to offer some kind of stipend to make it easier for students to participate. We are even looking at how Red Hat can itself be one of these sponsoring organizations, outside of the internship program Red Hat continues to run. Thank you, Google, for getting us started. We will take it from here.
Comments from this post were discarded during a website migration.
]]>I have found that the root cause is an infection of %System32%\drivers\atapi.sys, and that replacing this file with a clean version will get the system booting normally. This is not the first time that an infection hitting atapi.sys has caused updates to trigger bluescreens. If you are running Windows and have not yet applied this update, make sure you scan your computer thoroughly for infections before applying this update. If you are experiencing this problem, get your computer to a professional that can replace the infected atapi.sys and clean any other malware from your computer.
References:
http://isc.sans.org/diary.html?storyid=8209
http://social.answers.microsoft.com/Forums/en-US/vistawu/thread/73cea559-ebbd-4274-96bc-e292b69f2fd1
1. Boot from your Windows installation CD Insert your Windows installation CD and boot your computer. If your computer is not set to boot from CD first, you may need to reconfigure your BIOS or press a boot menu key (often F12, F8 or Esc). If you are unsure of how to do this, consult your favorite geek. As soon as the boot starts, you should see a message like "Press any key to boot from CD..." - press a key.
2. Start the Recovery Console After the CD loads (it may take a minute), you will be presented with a few choices. One of these options is to start a recovery by pressing "R". Press "R" to launch the Recovery Console.
* You may be asked to choose a Windows installation. If so, choose the damaged installation (probably "1").
* You may be prompted for the Administrator password. If you do not have one, press "Enter".
3. Identify your CD drive letter You should now be at the command prompt. Enter the following command:
map
Look for the drive letter for your CD drive. It may look something like this:
D:\Device\CdRom0
In this case, your CD drive is "D:".
4. Replace ATAPI.SYS Enter the following, replacing "D:" with your CD drive:
cd system32\drivers
ren atapi.sys atapi.old
expand D:\i386\atapi.sy_
You should see the message "1 file(s) expanded." - this indicates you have succeeded.
5. Reboot and scan for malware Reboot your computer. With a little luck, your computer will now boot normally. Because this problem is caused by malware, you should immediately scan your computer with up-to-date antivirus software.
UPDATE:
An atapi.sys infection may not be the only cause of this blue screen. While it does seem to be the most common cause, other infected drivers or drivers that make incorrect references to the updated kernel bits may also cause blue screens after this update is applied. Make sure you scan any computer with up-to-date antivirus software that can detect rootkits and check for updated drivers for your computer before applying this update.
UPDATE 2:
I have placed these instructions on my wiki. Any further changes will be posted there.
Comments from this post were discarded during a website migration.
]]>We have been extremely lucky so far. Most ransomware uses vulnerable encryption, like a simple XOR cipher, or a common key that need only be compromised once and then distributed to affected users. The distribution of ransomware has also fallen short of that of other threats like scareware. The number of people affected by ransomware has so far been small, and security researchers have been able to distribute unlocking tools capable of defeating the ransomware, but how long will we be so lucky? It may only be a matter of time until a more sophisticated, widespread ransomware assault hits the ill-prepared.
When ransomware uses strong encryption and uses unique keys for each victim, security researchers may be unable to offer unlocking tools and a victim's only recourse would be to pay the ransom. When a widespread attack hits, the damage could be devastating, and the returns for the attackers would certainly provide inspiration and funding for further attacks. Some ransomware uses SMS short codes to take payments, which may allow attackers to hide the final billing amount or apply recurring charges and may allow panicked and unsuspecting minors to unknowingly make the payment without first alerting their parents. Introducing mobile providers into the mix may also affect the ability of the victim to recover the charges.
Scareware distributors have already figured out successful models, their threats are already close to the behavior of ransomware, and they certainly have the resources to develop more advanced ransomware. Perhaps scareware can serve as a preview into what ransomware may do in the future. The protection is the same simple, long-standing advice: backups. If the important stuff is backed up, then a computer infected with ransomware can be cleaned and returned to service without concern for encrypted files.
If you find yourself infected with ransomware and do not have backups, find a computer service company with security expertise that may be able to recover your locked data. If you have paid the ransom, notify your credit card company or mobile carrier and get your computer cleaned by a professional as quickly as possible. If you do not have a backup routine, now is the time to create one. You may be on borrowed time.
Comments from this post were discarded during a website migration.
]]>Application whitelisting allows an administrator to restrict what programs may run on a computer to a trusted list, instead of a normal configuration where all programs are allowed unless explicitly blocked. This is a highly effective way to block malware and unwanted programs from being installed or used on the system. The configuration outlined below is ideal for office setups where most users run as restricted users and only a few people may have administrative access. When set up on a domain, this can dramatically reduce the threat of malware, but it does take a lot of freedom of choice away from the users and increases the administrative burden. Running as an administrator bypasses the protection of this configuration. Windows XP and later have built-in support for application whitelisting. This support is not as robust as that provided by third-party application whitelisting products, but can still be used effectively. Prior to Windows 7, this feature is available as "Software Restriction Policies". In Windows 7 and later, it is available as "AppLocker".
Software Restriction Policies is a system policy feature that can be configured through local system policies or through Group Policy. To configure Software Restriction Policies as part of Group Policy:
With these settings, administrators will continue to be unrestricted, while regular users will only be able to run programs that have been installed under the Windows installation folder (typically "C:\WINDOWS") or under "Program Files". They will be able to launch those programs using shortcuts. Administrators must handle program installation on behalf of the system's users.
When adding rules for network shares and mapped drives, you must use the UNC path in the rule. Using the mapped drive letter will not work.
More information on Software Restriction Policies is available from Microsoft TechNet at: http://technet.microsoft.com/en-us/library/bb457006.aspx
Comments from this post were discarded during a website migration.
]]>Since almost everyone on microblogging services is using the shortening services, it is impractical to avoid them or to blacklist the associated domains. There are ways to safely extract the end URL from the short version, but those methods are not currently readily available to the majority of users. Bit.ly, one of the shortening services, recently introduced warnings when they detected that a URL target might be malicious or intended for unsolicited use (spam). This is a start, but it is still retroactive. TinyURL offers, through a setting that users can enable (stored in a browser cookie), the option for users to see a preview of the target URL before being redirected. Bit.ly offers a Firefox extension for the same purpose. These options are better, but they still require that the end-user take some action for the extra protection. For administrators of sites like Twitter, there are very few options for screening shortened URLs.
Because of the variety of shortening services available, and the ability for users to pick from any of them (and post any arbitrary URL), the only way to dereference the URLs in a widely-supported manner is to actually attempt a simple HTTP request to each posted URL and, if the response code is 30x, then look at the "Location" response header. Thus far, I have seen no evidence of a major microblogging service doing any filtering on shortened URLs, but I would not expect them to disclose their anti-spam measures if they did. Since malware and spam attacks are targeting microblogging services with increasing frequency, filtering and blacklisting of URLs may soon become a necessity.
I can imagine two things that would go a long way toward protecting users from the increased threat and return the balance. First, browsers should include support for dereferencing links without visiting the targets, actively notifying users of the target URL when they are being redirected, and have that feature enabled by default only where the target is on a different domain. Second, microblogging hosts should introduce filtering of shortened URLs by checking all links posted on their services for redirects and then filtering those redirection targets, and they should coordinate blocking efforts to increase the effectiveness of the filtering.
Spammers are already taking advantage of shortened URLs, and the problem is only going to get worse unless we take action to destroy the advantage that shortening services currently give them.
Update: The suggested browser feature from above has been proposed for Firefox at: https://bugzilla.mozilla.org/show_bug.cgi?id=453077Comments from this post were discarded during a website migration.
]]>In Amarillo, T-Mobile does not yet have 3G service, so data speeds leave a bit to be desired, but it still works. I learned earlier this evening that I may be taking a trip to San Antonio in the near future, unfortunately not on pleasant business. I know that I will get 3G service there, and I do intend to take my notebook, but I do not have a cellular card for my notebook, and it is an old model that does not have built-in wireless. Getting Internet to my laptop on the go has always been a hassle - but what if I could use my G1 to get Internet to my laptop?
It did not take much searching to find Tetherbot, which provides some basic tunnelling for a computer connected via USB to an Android-powered device. Using Tetherbot and the Android SDK, I was able to establish a tunnel for Internet browsing within a couple of minutes. Now I just need to finish upgrading my notebook to Fedora 11 (a hassle of its own) and I will be ready to go!
Comments from this post were discarded during a website migration.
]]>Wednesday, a tool was released that changes that for servers running Apache, Squid or any one of several other HTTP servers and proxies. This tool is able to bring down these servers by forcing them to open a large number of processes and keep the connections open using only a minimal amount of bandwidth. This means that an attacker with a low-bandwidth connection may be able to bring down a server on a much higher-bandwidth connection. A distributed attack using this technique could be absolutely devastating, even to larger server farms. The attack is very simple, much like many other DoS attacks.
Unfortunately, there are very few mitigation techniques known at this time, and like other DoS defenses, many of these techniques require balancing security and accessibility. Looking at this attack, I can imagine small variations that could potentially affect any HTTP server. I do not mean to cause alarm, but I strongly suspect we will see many more attacks like this one targeted at the major HTTP servers and able to bring down those servers with far fewer resources than attackers currently need to pull off a successful assault. Similar attacks may also be possible against other types of servers, such as mail servers or remote administration servers. If you are interested in testing against your own servers, you can find the tool at:
More discussion can be found at the SANS Internet Storm Center:
Comments from this post were discarded during a website migration.
]]>I have already changed "nman64" and "n-man" in many places to "patrickwbarnes". I have registered patrickwbarnes.com and redirected n-man.com to it. It will take some time to get everything changed over, and I intend to keep old contact information working for the foreseeable future. I will continue updating the links on this site and the information on the "Contact" page as the changes continue.
With the change in my domain name, I decided there was no better time to replace my old, outdated website with something new. I have been thinking about what to replace it with for some time, and I settled on a WordPress blog. I have tried blogging several times before and have always let it fail. Lately, there have been several times that I have thought about writing something, but have not had a blog to write it on. Since I am consolidating my website into a WordPress blog, perhaps that will give me more incentive to maintain it this time. Only time will tell.
Comments from this post were discarded during a website migration.
]]>