Moved to Wordpress

Here....

Mildly surprising

French fighter planes grounded by computer virus

"French fighter planes were unable to take off after military computers were infected by a computer virus, an intelligence magazine claims...The aircraft were unable to download their flight plans after databases were infected by a Microsoft virus they had already been warned about several months beforehand. At one point French naval staff were also instructed not to even open their computers. Microsoft had warned that the "Conficker" virus, transmitted through Windows, was attacking computer systems in October last year, but according to reports the French military ignored the warning and failed to install the necessary security measures. The French newspaper Ouest France said the virus had hit the internal computer network at the French Navy."

Good for a few laughs...this particular virus was supposedly primarily a threat to teens clicking on links to pirated music.

"The Mind Has No Firewall"

"Timothy L. Thomas, the Foreign Military Studies Office's tireless chronicler of OPFOR information operations, wrote an interesting article in Parameters about Russian information military theory...Thomas' major point was that while US IO theory seeks to dominate information systems, the Russians see the human mind itself as an information-processing system to be hacked."

The US military establishment would do well to study closely Russian strategic and tactical ideas. This example in information technology is instructive...the concept that technology is a tool to be used to influence people rather than an end in itself is correct. Americans tend to focus too much on technology itself, and not enough on the prospective users of technical systems.

Map_Reduce in Ruby: Event Machine

The linked piece describes yet another possibility involving "EventMachine, a library which makes efficient network programming relatively simple in Ruby". The key to this technique is apparently that "Map processes can be EventMachine servers. We can have an arbitrary number of those running on a number of physical nodes."

The code from the link:

module Map
def receive_data(path)
 document = File.read(path)
 word_counts = document.split(' ').map { |word| [word, 1] }
 send_data(Marshal.dump(word_counts))
 close_connection_after_writing
end
end

EM.run {EM.start_server("localhost", 5555, Map)}

class Reduce < EM::Connection
 @@all = []

 def initialize(*args)
   super
   @doc, @data = args[0], ''
 end

 def post_init
   send_data(@doc)
 end

 def receive_data(data)
   @data << data
 end

 def unbind
   Reduce.job_completed
   @@all += Marshal.load(@data)
   unless Reduce.pending_jobs?
     groups = @@all.group_by {|word| word[0] }
     groups.each { |g| p "#{g[0]} : #{g[1].size}" }
     EM.stop
   end
 end

 def self.send_map_job(port, doc)
   @job_count ||= 0
   increment_job_count
   EM.connect("localhost", port, Reduce, doc)
 end

 def self.increment_job_count
   @job_count += 1
 end

 def self.pending_jobs?
   @job_count != 0
 end

 def self.job_completed
   @job_count -= 1
 end
end

EM.run do
 {
   5555 => 'docs/america.txt',
   6666 => 'docs/da-vinci.txt'
 }.each { |port, doc| Reduce.send_map_job(port, doc) }
end

I'll update after I test this.

Map_Reduce in Ruby: Skynet or Starfish?

Skynet:

"The MapReduce design pattern to distribute data processing was introduced by Google in 2004, and came with a C++ implementation. A new Ruby implementation is now available under the name of Skynet released by Adam Pisoni."

Starfish:

"After a quick look at Starfish I decided that it was what I would attempt to use to accomplish this task. It's advertised as a simple Ruby version of map reduce by its creator Lucas Carlson. I like how simple the interface is and it would be perfect for my needs if it weren't for one huge problem. The memory usage of the processes it kicks off are huge (something like 30 MB)! I assume this is because of ActiveRecord. Unfortunately this eliminates Starfish/ActiveRecord as an option since the VPS I'm setting the prototype up on has only 256 MB of RAM.

I have a few other observations on Starfish. It's really only the map part of map-reduce (which is still quite useful). Also, I don't quite understand how to start up multiple clients with it. The documentation says to just run:

starfish my_awesome_map_reduce.rb

I guess that starts the whole thing and then running that again will kick off additional clients. I need to dig into the starfish code to more closely understand things, but that didn't seem like a decent way to kick off multiple clients. The issue I had is that if you're kicking them off using a rake task, how can you be sure to start up the additional clients after the server has successfully started? Here's the super lame thing I did:

namespace :crawl do
desc "Kicks off Starfish 3 times for feed_updater.rb"
task :feed_updater => :environment do
method(:fork).call { system("starfish lib/feed_updater.rb") }
sleep(30)
2.times do
method(:fork).call { system("starfish lib/feed_updater.rb") }
end
end
end

Of course that was just to test things and now that I know these processes are whores for memory, the whole strategy is out anyway. I am curious how other people using Starfish kick off the additional clients."

Plus an interesting comment regarding Google's implementation from the second link:

"As a final aside, Lucas mentions how distributing the task of processing a log file slowed things down by 20x because of the problem of communicating and sending the data over the wire. I assume MapReduce would have this problem too if it weren't for the Google File System (GFS). One of the things the paper mentions is the importance of taking advantage of locality of data. Meaning that calls to process certain chunks of the input are put to machines that have a cached copy or are in the same rack as the machine that holds the data. The GFS paper also mentions data redundancy across the cluster which is something I would think might help MapReduce run faster."

Here's an example of Starfish in use from its author:

require 'config/environment'
require 'user'
require 'notifier'

server do |map_reduce|
map_reduce.type = User
map_reduce.conditions = "opt_out = 0"
end

client do |user|
Notifier.deliver_email(user)
end

This tiny amount of code with next to nothing that needs to be
memorized and takes 30 seconds to write down can potentially save you
hours in deliver time. Even running 10 clients at once on the SAME
MACHINE gave us nearly 10x the speed it would have taken serially. This
was not mission critical, but gives you a good sense of ways to apply
Starfish to mission critical applications.

Using simple_http to simply grab a web page

Thinking out loud in Ruby:

require 'simplehttp'
data = SimpleHttp.get "www.example.com"
afile = File.new('c:\testfile.htm',"w+")
afile.puts data
afile.close

A start at automating OpenOffice with Ruby

I found this code here, which was written in 2004 and worked today:

require 'win32ole'
objServiceManager = WIN32OLE.new('com.sun.star.ServiceManager')
Stardesktop = objServiceManager.createInstance('com.sun.star.frame.Desktop')
doc = Stardesktop.loadComponentfromUrl('private:factory/swriter', '_blank', 0, [])
text = doc.getText()
text.setString('Hello World')

This appears to be the current documentation for OO's automation functionality:

:: com :: sun :: star :: bridge ::

Launching a Firefox instance-Ruby

Very simple; use the system() command, this launches Firefox with whatever your default home page is:

system('C:\Program Files\Mozilla Firefox\firefox.exe')

Here's another way to do it:

foo = IO.popen('C:\Program Files\Mozilla Firefox\firefox.exe', "r")

The popen method runs a command as a subprocess and connects the subprocess's standard input and standard output to a Ruby IO object. The connection isn't really useful in this example, but would be in other cases.