Donnie Demuth

Bioinformatics with Python and Hadoop Streaming

2010-07-08T06:38:00.000-07:00

A while back I wrote a short paper about using the Python programming language in conjunction with Apache Hadoop. The purpose of doing so was to use several to many heavy duty machines to process problems in the field of Bioinformatics. At the time, I did not come across many resources so I've decided to post some of my work on the web. The intended audience would be students in an introductory Bioinformatics course or anyone with a will. It's really simple stuff!

The Paper/Tutorial:
http://donniedemuth.s3.amazonaws.com/bioinf_with_python_and_hadoop.pdf

Relevant Slides:
I also gave a presentation on functional programming, Python, and Hadoop. The slides relate heavily to the paper above.
http://donniedemuth.s3.amazonaws.com/LCS_python_hadoop_streaming.ppt

Note:

I haven't edited the material yet so there may be grammatical errors. And eventually I plan on moving the contents onto the blog.

CSCI E-190 Bioinformatics Algorithms

2010-02-03T10:23:00.000-08:00

CSCI E-290 is a fairly new course in the Harvard Extension School curriculum related to three different programs: biology, computer science, and biotechnology. I'm taking it as an elective in my pursuit for a Masters in Liberal Arts (ALM-IT) with a concentration Software Engineering; though after a little bit of studying I'm curious why I never pursued the biotechnology path.

Note: When I was younger, I was terrible in the sciences (biology, chemistry, you name it). Actually, if it wasn't computer related it couldn't hold my attention.

The following are my impressions after the first lecture.

How was the lecturer?
Dr. Jeff Parker's lecture opened up to the following youtube clip:

I think it was a tell tail sign that he has a sense of humor and that the class will not be dry. Throughout the lecture I was certainly entertained. Surprisingly the pace of the lecture was very fair and of the 100 slides he had in his PowerPoint, we only covered fifty-or-so.

One thing that I noticed about the lecturer was that he seems very patient. A few students blurted out ridiculous questions, in-my-opinion not appropriate for a first lecture, and he took a deep breath and tried to answer them fairly. Personally, I don't know if I could ever be that calm. I understand that everyone learns in different ways but I wish people would respect those trying to educate them.

Overall Impression
It is hard for me to judge this class for the following reasons:

I love the Python programming language
This class uses Python
I am fascinated by what I'm learning

Thus, I am currently geeked out and favorable towards the class. If you want a fair look at the class, you may have to stay tuned for a later blog post. I plan to review some of the required course books (I bought many) and workload.

CSCI E-250 Abstraction and Design [First Impressions]

2010-02-01T12:04:00.000-08:00

CSCI E-250 is the Harvard Extension School's offering of the Harvard College's CS-51: Abstraction and Design. There are two reasons I signed up for the class: a really good teaching fellow and I am always interested in becoming a better programmer.

The following are my impressions after the first lecture.

How was the lecturer?
The course is taught by Professor Greg Morrisett. He is not dull by any means, speaking with authority and passion (through his highly-validated opinions). For example, if you're a C++ programmer you could be offended by some of his opening remarks -- apparently, teaching the C++ would go against everything he believes in. Luckily, I avoid C and C++ like the plague so I think I like this guy already.

One statement made me very interested in the class. He states:
"[The class will] change the way you think about programming in a deep way."

A few more times he mentions that there will be an emphasis on stretching of the mind. The stretching is probably related to learning the functional programming language OCAML. And well, this is all exciting. I think I ought to look at programming in a different light.

Why OCAML?
In undergrad I studied Haskell, an older functional programming language, and I thought it was the best language ever. However, after that particular class I never used it again. I don't remember a thing about it except I was probably the only person that got an A in the class. My peers highly despised the language. It should be fun learning the concepts again.

In the Spring '09 edition of this course we'll be using OCAML and a few given reasons are:

Functional programming languages are better
FPLs are used on Wall Street and for huge problems, see Map-Reduce
New FPLs are on the rise: F# and scala
Languages such as C++ are error prone no matter how skilled the programmer is
It uses one of the highest-performance compilers out there
It is mostly pure; few side-effects

After our first look of OCAML
The syntax of OCAML does not scare me a whole lot. I recently took a Ruby course so I have a little experience with the concept of a map function. However, a few simple things stood out:

Two Semi-colon symbols are used to end a statement or function
The underscore symbol matches any string
The in keyword can be viewed as a way to define multiple let statements within a block of code

Around the time the professor posed the question about why nested comments were important. If you ever programmed with Java you would realize that you can really nest comments. What occurs in some languages is that the inner comment-close syntax will also close an outer comment.

Update: I found a decent tutorial in written for the layman.
Update-2: Here's another site I've never seen before. The code-codex has comparable code to many different languages and it seems like it could be a decent learning tool.

Overall Impression
Simply put: it was pretty darn cool. Even this link was thrown within the slides:

http://www.malevole.com/mv/misc/killerquiz/

It's a game that asks you determine whether you determine if a person, by profile picture, is a serial killer or programming language inventor.

Not a lot of Extension students are taking this course. I DO NOT know why. It seems like this class is meant for them (me). I leave you with this statement from the lecturer: The class is intended for practicing software people and will to help you "kick ass" in the work place.

CSCI E-168 Web-based Software with Ruby and Ruby on Rails [REVIEW]

2010-01-13T12:07:00.000-08:00

Update: Since I've posted this, I received several e-mails about the course. One included a course evaluation that may be relevant. I only included the comments of this eval and I am posting it anonymously for the person.

A fellow classmate reminded me to review the CSCI E-168 course. Here is my review!

Lectures
The lecturer, John Norman, is articulate and an incredible speaker. You can immediately tell he comes from an English-Literature background. His lessons are pleasant to listen to and you can literally hear the pride that puts into his work. I believe most people would enjoy his lectures.

The Ruby portion of the course was amazing. I felt that this was a language I could work with and the Ruby-only assignments were really fun.

Sections
Unfortunately, I had a scheduling conflict and could not stay for section most of the time. Since the class had over 70 students, there were ~~two~~ three different sections and YMMV.

Update 1-16-10:
Okay so there seems to be some misunderstanding whether or not I liked section. I was questioned about it in several private e-mails. In my past posts, I kind of gushed over class and section.

Of the three classes was taking it was the least useful of the group. However, it is worth mentioning that I did read ahead and finish many assignments earlyDid I need to go to section? Probably not. But I thought it was fun, and funny at times. For the most part I sat back and kept my mouth shut.

Workload
I thought the assignments were moderately difficult. None were easy and none were outrageous. If you devote a good ten hours or so, you will be able to finish any assignment. However, a good portion of the assignment grades are based on how rubyish your code is. You will get slapped on the wrist for not doing something the ruby way. Keep that in mind and be sure to comment and document everything.

One thing that made me sour was trying to figure out what to do for the assignment. The documentation is wild and it's not clear what to do. I really don't have any tips for this class except for re-iterating that you have to do things the ruby-and-rails way. The following link is my final assignment submission for Fall 2009: E-168 Final Submission

My project was an attempt at a discussion board system. You may be able to review the code to determine what you can or should not do.

Final Thought

* Warning: Any negativity below is primarily Rails (the framework) related and not directly course related

CSCI E-168 reminded me why I am not a Ruby-on-Rails guy. In a nutshell, the course was a lot like going to Church. It's all interesting but I really don't know if I can buy into this...

Basically, there was a lot of cool things to learn but I'm not sure I would add it to my programming utility belt. As a career introvert the Rails community hits me in the wrong spot. There are far too happy go-lucky and bubble-gummy. It's the pop-music of web frameworks. A guy like me wouldn't fit in. What's wrong with that?!

This is an exaggeration but it also feels like a cult... An exclusive cult. Not like a Unixy-nerdy cult. It's a trendy iPody cult.

One gripe I had about the class is the guest panel at the end of the semester. Some incredible Ruby developers showed up: Dan Chak, author Dan Croak, and the owner of ravelry.com. They were invited to share their experience with the class. That sounds exciting right?

But from my point of view, I felt that the developers were not happy to be speaking in front of our class. Their mannerisms showed that (a) they were not getting paid and (b) why am I here? Moreover, the panel discussion was dominated by a previous Teaching Assistant who had very little to add; except for cutesy sarcastic jokes. When this person spoke, I found entertainment by watching the expressions on the two Dan's faces.

I would recommend this class to someone that wants to work with Rails. Throughout the semester, I received many Rails-related calls from recruiters. You may experience the same interest from the industry.

Additionally, this class changed my opinions about Ruby. That language is pretty darn cool and the one-liner assignment is one of my favorite assignments of all time. In 2010, I heard that Harvard Extension will be offering a Programming Ruby course and I would put my personal stamp of approval on it!

CSCI-E 207 Formal Systems and Introduction to Computational Theory [REVIEW]

2010-01-02T19:30:00.000-08:00

This course made me experience Stockholm Syndrome. It held me captive and consume every waking moment of my life. For that, it became everything to me.

Lectures
It was difficult to find the time to watch or attend the lectures. When I took this course I was: simultaneously enrolled in three classes total, searching for full-time employment, and juggling a consulting gig. BAD IDEA!

The homework was so difficult that by the time you finish one... A brand new and much harder one is already posted online. Sure, lecture was important to understanding the material but it only covered 1% of what you'll have to learn in order to complete the Problem Sets. Well, it just felt that way!

Unfortunately, the lecturer could only cover so much in an hour and a half. Lecture provided an overview but section provided what you needed to know to get started on your homework.

Sections
Watch or attend section, ~~or else~~. If you don't, you'll do poorly. I guarantee it! Brian was our Teaching Fellow and he was the best TF you could ever ask for. He never gave us the answers -- but I am certain he could teach chimps poetry. If you have a TF half as good as him you're in good shape. Often I felt section covered the material very clearly, much more than the book or lecture.

Workload
You'll have problem sets almost every week and they'll take about 20 hours or more to complete. This class seemed to be a "re-take" for others but this was the first time I covered this material. If you're taking this as a refresher, then sure, I guess it won't be that hard. But for me, it made me cry myself to sleep.

Besides the weekly homework, you'll have a midterm and a final. I studied diligently for them so I did pretty darn well. Both were difficult -- BUT not as difficult as I thought they would be.

You will also need to spend a week or two learning LaTeX and its markup language. LaTeX is intimidating at first and it does take some time to install. To get my MacBook set up correctly it took me over three hours. Most of the time was due to download a large gigabyte file. I believe there's some dmg you can download to set LaTeX all up for you. The markup language isn't too difficult but it does take a while to get used to as it's not intuitive (at first). On a MacBook, I found TexShop to my editor of choice.

I recommend anyone taking this class to download a tool named JFlap to help you with 2 or 3 of the Problem Sets. Trust me, JFlap is necessary and will help you test your answers. Drawing automaton or construct grammars is a piece a cake. Testing it is just as easy as your can run multiple input strings against your object (as shown above).

The assigned textbook is Introduction to the Theory of Computation (Sipser) and it's good but terse. I used the first edition because it was whole lot cheaper and I didn't have any problems. For the homework, you'll probably need more help if you can't figure things out on your own. A few books that helped me were:

How to Prove it (Vellemen): This book definitely helps for the first Problem Set and will help you write proofs for the rest of the semester. I came back to this book a few times to remind myself how to solve later problems in the course. It costs about $20.

Link to How to Prove it on Amazon

Introduction to Automata Theory, Languages, and Computation (Hopcroft, et al): I don't know why this book doesn't have a higher rating on Amazon. Out of all of the books I've used and looked at, this was by far the best for this class. I didn't find this until the end of the class and read it from cover to cover. If I had it early, many of the Problem Sets would have made so much more sense... SAD!

Link to Automata Theory on Amazon

Final Thoughts
The class was a roller coaster ride: I loved it, hated it, loved it, hated it, and so on. I ended up getting an A-minus but I know I earned it. If you want the same, work hard and treat this class like a full-time job. I'd take it again just to get that A.

CSCI-E 131b Communication Protocols and Internet Architectures [REVIEW]

2010-01-02T18:12:00.000-08:00

Now that I have completed the class I ought to tell you all about what I thought.

Lectures
As many people will tell you, Leonard Evenchik is an excellent lecturer. I enjoyed his talks and I never felt "lost" even when I didn't understand the material. He's a great guy and although I did not have much one-on-one time with him I'm sure he would be fun to talk to.

If you can't attend class you can watch his recorded lectures online on the course website. I was pleasantly surprised to find a quality recorded product. The lectures will stream at various sizes and include the power point slides he is covering. If you're lucky enough to take his class when it's held at 1 Story Street, you'll have the better product to watch (many more camera angles).

This class is recorded and held live in the Fall and the Summer. The videos are replayed for those taking it in the Spring. Apparently the Summer class is a little experimental where the staff tries to introduce new technology.

Sections
When I took the course "Joe" was my TA and only 5-7 students showed up to section each week. Joe is a really good guy who had a sense of humor and encourage everyone to talk and share our newbie questions (the stuff covered in the class does seem silly for us not to know). I spoke a lot and I was surprised that the other students had the same questions that I had. An added benefit was Joe really helped us understand the homework problems and past lectures.

Workload
Okay, so if you're thinking of taking this class I bet you just want to know how hard is it? For me it was not difficult. But I did find the material interesting and the homework fun. Each homework might take 5 to 10 hours to complete and there are 5 of them.

You'll spend a few hours reading each week and most of it will be online material and relate to the homework. The assigned textbook Computer Networks, A Systems Approach (Peterson and Davie) is good but seems to lose importance during the middle chunk of the course.

I found that the book TCP/IP Illustrated, Volume 1: The Protocols (Stevens) covered that section better and is a better supplement for many of the homework problems. The problem with the later book is that it does not cover many of the modern technologies related to VOIP. Hint: Professor Evenchik is a VOIP expert.

However, you can buy that book on Amazon for $50 new, used for $20.
http://www.amazon.com/TCP-IP-Illustrated-1-Protocols/dp/0201633469

Final Thoughts
I enjoyed this class and I would recommend it to anyone interested in the IP stack and networking. It is ridiculous how much better I understand the Internet. If you're already comfortable with TCP/UDP/IP and all those other three-letter-acronyms then this class could be extremely easy. But for a guy like me, with no in-depth networking experience, I found this to be the most useful class I have ever taken.

CSCI-E 131b Final Review

2009-12-21T13:17:00.000-08:00

Update 1/2/10
The Final didn't seem as difficult as I thought it would be -- I did study a whole lot and attend every section though! If you're taking this course in the future, I recommend having a good grasp on ALL concepts. Easier said than done right? But guess what, your brain will have a lot of extra room if you don't memorize the structure of specific packets and other things of that detail.

~~~*~~~*~~~*~~~*~~~*~~~*~~~*~~~*~~~*~~~*~~~*~~~

Note: I formed this while I was studying for the course’s final and I cannot vouch that it is correct in any way. Much of it comes from our course’s textbook, Wikipedia, lecture notes, and my own random knowledge.

Terms

These are only the terms that I think are important.

802.3: The IEEE Ethernet standard used today.

802.11: The Wireless networking standard used today.

ACK: An abbreviation for acknowledgment. ACKs are usually sent to tell the sender the data or packet transfer was successful.

AES: Advanced Encryption Standard. A Shared-Key cipher that supersedes DES.

ARP: Address Resolution Protocol. Used to translate IP addresses into MAC addresses.

Bandwidth: The amount of data per time that can be transmitted over connection.

Broadcast: A way of sending packets to every host on a network.

CA: Certificate Authority. This is an entity that verifies and signs certificates to ensure the validity of a public key and name (domain).

Certificate: A digitally signed (hashed) document used to distribute public keys.

Checksum: A computation over some data that can take place before and after transmission. If the checksum matches, the before and after, then the data was transmitted in whole. Used for error detection.

CIDR: Classless Inter-Domain Routing. Used for sub-netting, allowing many more sizes of networks compared to typical Class A, B, C subnets.

Congestion: The occurrence packets are discarded when too many contend for a single resource.

Congestion Control: The avoidance of congestion in a network. We’ve discussed the Slow Start algorithm of TCP in class.

Connection-Oriented: A protocol where some initialization must occur between the sender and receiver before data may be transferred.

Connectionless: A protocol where data can be sent without any prior connection. Also known as a Datagram service.

CRC: A strong checksum that exists in many packet headers.

CSMA/CD: Carrier Sense Multiple Access with Collision Detection. This is a feature of Ethernet. It can tell when data being sent over the network as multiple nodes can be attached to it. In addition it is aware when more than one entity transmits data at a time.

Datagram: Analogous to connectionless. This is a transmission unit that contains the necessary information to deliver to its destination.

Demultiplexing: The counterpart to multiplexing, where many different “things” can share another “thing.” As in the case of protocols, IP uses the Protocol Number field to determine whether it is using TCP or UDP. And TCP/UDP uses the Port Number to allow many Layer-5 protocols to use it.

DES: Data Encryption Standard. This is a Shared-Key algorithm that uses a 64-bit shared key.

DHCP: Dynamic Host Configuration Protocol. A protocol used by a host to determine it’s own IP on a network.

DNS: Domain Name System. The naming system used by the Internet to resolve hostnames, implemented through a hierarchy of name-servers. Common DNS records include:

• A -specifies 32 bit IPv4 address

• AAAA –IPv6 address record

• MX -mail exchange record

• NS -specifies authoritative name server for a domain

• CNAME -canonical name, provides alias functionality

• HINFO -specifies limited host information

• SRV –identifies a specific service

• NAPTR

Encapsulation: The process of taking a higher level protocol and placing it within the payload of a lower level protocol.

Ethernet: A data layer protocol that uses CSMA/CD. The original Ethernet had various hosts that used vampire clamps to connect to a large wire, “the ether.”

Firewall: A router that follows some security policy to filter packets.

Flow Control: Used to prevent a sender from overloading a receiver. We most commonly see Sliding Window as a mechanism in HLDC and TCP.

Forwarding: Routers operate on the store-and-forward principal. Packets are first stored in a buffer and then sent to its destination.

Forwarding Table: Maintained by routers to help decide where to forward packets.

Fragmentation/Assembly: Packets may be split into small sizes by a router if they are too large for a network.

Frame: A (link-layer) name for a packet sent between two links.

FTP: File Transfer Protocol. A standard TCP based protocol used for transferring files.

H.323: A protocol used for Internet telephony. Assumes that the end devices are “simple” and provides more control and configuration over SIP.

ICMP: Internet Control Message Protocol. Allows reporting based on the IP datagram, as IP is connectionless and occasionally some response can be helpful to certain applications/protocols.

IMAP: Internet Message Access Protocol. Allows a user to access their mail without downloading it to their machine first.

IPSEC: IP Security. This is an architecture used to provide authentication and security to the IP layer of the Internet. Transport (encrypt only the data) and Tunnel (encrypt all, assigning a new header) modes are provided.

Jitter: Timing variations in network latency.

MAC: Media Access Control. We see this as a way to share single device with a common network.

MD5: Message Digest version 5. This is a Digital Signature/Hashing algorithm.

MIME: Multipurpose Internet Mail Extensions. Email was original text-based and MIME provides a way to convert and specify binary data to text.

MTU: Maximum Transmission Unit. The largest sized packet that can be sent on a given network.

Multicast: A special form of a broadcast to send data to a specified group of nodes.

Multiplexing: A way to share a single resource. Both UDP and TCP use the IP protocol to send data over the net. Examples:

FDM – Frequency Division Multiplexing – A different frequency for each user.

TDM – Time Division Multiplexing – Time intervals for each user or Statistical by a queue.

NAT: Network Address Translation. Typically implemented by routers to assign outgoing traffic from a local address some known public address so it may access the Internet. NAT may use port numbers where multiple local hosts share a single public address.

NFS: Network File System. A protocol to make file access over a network appear transparent.

OSPF: Open Shortest Path First. Used by routers to construct a network topology and be aware of changes noticed by other routers.

Packet Switching: Is the term used to describe how data is sent through the network. It uses store-and-forward and implies statistical multiplexing.

Proxy: An intermediate machine between a sender and receiver which can intercept messages and provide some service.

Public Key Encryption: An encryption algorithm where users have a private and public key used to encrypt and decrypt messages, versus some shared key. The private key can encrypt a message that only the public key can decrypt.

QoS: Quality of Service. An implementation of QoS can allow a network to make guarantees on packet delivery. Certain packets may be marked for expedited delivery.

RIP: Routing Information Protocol. Each router only aware of its networks and forwards this information to other connected routers.

RSA: A public-key encryption algorithm.

RTCP: Real-Time Transport Control Protocol. RTCP provides out-of-band statistics and control information for an RTP flow.

RTP: Real-Time Transport Protocol. RTP is an end-to-end protocol used to send data with real-time constraints. This is unreliable but sequenced.

RTT: Round-Trip Time. This is simply the latency to reach the destination and back.

SDP: Session Description Protocol. This is a format for describing streaming media initialization parameters in an ASCII string. SDP is intended for describing multimedia communication sessions for the purposes of session announcement, session invitation, and parameter negotiation. SDP does not deliver media itself but is used for negotiation between end points of media type, format, and all associated properties. SDP is designed to be extensible to support new media types and formats.

SIP: Session Initiation Protocol. This is an application layer protocol used in multimedia applications. It determines the correct device with which to communicate to reach a user, determines is the user is willing or able to partake in communication, determines the choice of media and coding scheme to use, and establishes the session.

Sliding Window: Sliding Window Protocols are a feature of packet-based data transmission protocols. They are used in the data link layer as well as in TCP. They are used to keep a record of the frame sequences sent, and their respective acknowledgements received, by both the users. Their additional feature over a simpler protocol is that can allow multiple packets to be "in transmission" simultaneously, rather than waiting for each packet to be acknowledged before sending the next.

Slow Start: Slow-start is part of the congestion control strategy used by TCP, the data transmission protocol used by many Internet applications. Slow-start is used in conjunction with other algorithms to avoid sending more data than the network is capable of transmitting, that is, network congestion.

SMTP: Simple Mail Transfer Protocol. This is an Internet standard for electronic mail transmission across networks. For receiving messages, client applications usually use either the Post Office Protocol (POP) or the Internet Message Access Protocol (IMAP) to access their mail accounts on a mail server.

Sub-netting:

• Class A, networks 1 -126, /8 prefix

• Class B, networks 128 -191, /16 prefix

• Class C, networks 192 -223, /24 prefix

Private IP Addresses

• 10/8 10.0.0.0 to 10.255.255.255

• 172.16/12 172.16.0.0 to 172.31.255.255

• 169.254/16 169.254.0.0 to 169.254.255.255

• 192.168/16 192.168.0.0 to 192.168.255.255

TCP: Transmission Control Protocol. This is a connection-oriented and sequenced protocol that ensures the delivery of data. Some well known TCP port numbers include:

20,21 FTP

22 SSH

23 Telnet

25 SMTP

80 HTTP

110 POP3

1720 H.323

5060 SIP

UDP: User Datagram Protocol. This is a connection-less and un-sequenced protocol.

Virtual Circuit: Provided by connection-oriented networks where a connection is initialized, a virtual circuit is formed, and then data is sent.

VPN: Virtual Private Network. Provide some network tunneling between nodes and forms a virtual circuit. It has two modes, Transport and Tunnel. Transport – only the data/payload is encrypted. Tunnel – the whole IP packet (data and header) is encrypted, into a new IP packet with a new header.

SP3

SP3 is a framework for describing Protocols that we use solely in class. In this sections, I’ll cover some of the Protocols I think are important using SP3.

SP3: Guidelines

Service – What service is provided by this technology? For example: is data reliable, sequenced, or unreliable (connectionless), and what combinations of these features exist?

Purpose – What does this technology attempt to solve? For example: addressing, multiplexing, sequencing, error detection/correction, flow control, security, fragmentation and assembly.

Packets – Describe the (header) fields of the packet.

Procedures – What are the procedures to use this technology? For example: connection establishment, capability agreement, and data transfer.

PPP: Point to Point Protocol

Much of this is from lecture notes:

Service – PPP provides a connection-oriented service and, like HDLC, gives the physical layer the appearance of being an error-free link.

Purpose – To deliver the promised level of service, PPP is capable of encapsulating multiple-protocol datagrams, using a link-control-protocol for establishing, configuring, and testing the data-link connections, and using a family of Network Control Protocols (NCPs) for establishing and configuring different network-layer protocols. This provides framing, encapsulation, authentication, among others.

Packet – PPP frames look similar to the ISO HDLC standard. The fields contained are: flag, address, control, protocol, information, FCS, and flag. Each frame begins and ends with a flag field set to 0x7E. The address field is always set to 0xFF and the control byte begins at 0x03. The protocol field declares the type of data/payload is in the information field. The FCS is the frame check sequence used to detect errors in the frame.

Procedures – PPP, a reliable link layer protocol implements the following procedures: link initialization, link data transfer, link termination, and error handling.

HDLC

Service – HLDC provides a reliable Data Link layer service and as such, it gives the physical layer the appearance of being an error-free link.

Purpose – To deliver the promised level of service, a reliable Data Link level protocol such as HDLC must handle the following problems: Synchronization and framing, data transparency, data transfer, addressing, flow control, error detection, and error correction.

Packet –

Procedures – HDLC, a reliable link layer protocol implements the following procedures: link initialization, link data transfer, link disconnect, and link error handling.

802.3

Much of this is from lecture notes:

Service - 802.3 is an unreliable data link layer local protocol, where each device on a network may transmit data at it’s own discretion. 802.3 uses a logical bus configuration, and is well suited to a network with a light to medium load.

Purpose – 802.3 provides an unreliable level service to the Network layer with no acknowledgements or traffic prioritization. Error detection but not correction is provided with a checksum mechanism.

Packet – Preamble (7), Start of Frame Delimiter (1), Destination Address (6), Source Address (6), Length Field (2), Data (0-1500), Pad (0-46), Checksum/CRC (4)

Procedures – When a node wants to transmit data in 802.3, it listens to the physical cable. If the cable is busy, it waits until it is available and then tries transmitting again. If there is a collision during transmission, both nodes which were sending data immediately stop transmitting and wait a random amount of time before attempting to retransmit.

Ethernet

The Ethernet is very similar to its successor, 802.3, but differs in the packet definition. The Protocol Type of Ethernet was replaced with the Length Field. The Protocol Type can still exist in 802.3 as it is commonly the first bytes in the body.

Packet – Preamble (7), Start of Frame Delimiter (1), Destination Address (6), Source Address (6), Protocol Type (2), Data (0-1500), Pad (0-46), Checksum/CRC (4)

Frame Relay protocol

The following is from lecture notes:

Service - Frame Relay is a simple data link level protocol that provides a method to transfer data (frames) very quickly from one network point to another network point(s). It provides an unreliable service and it is used in networks where the physical layer communications lines are reliable and fast.

Purpose - The Frame Relay protocol is unreliable: it provides error detection but not error correction. It has minimal overhead, provides the address functionality that is required to deliver a frame via the use of a circuit ID called a DLCI, and it provides no flow control. There is very limited congestion control. When problems arise because such techniques are not implemented, frames that cannot be delivered are discarded. As a result of this lack of reliability, upper layer protocols must provide any necessary reliability.

Packet- A Frame Relay packet begins and ends with a flag character (7E hex). After the Beginning Flag character, the next bytes contain Addressing information used to transfer the packet across the link. Specifically, these bytes indicate which virtual circuit (DLCI) to use to route the packet and if the packet is eligible for being discarded (DE bit).

There are also bits to indicate whether or not the network is becoming congested (the Forward Explicit Congestion Notification bit - FECN, and the Backward Explicit Congestion Notification bit –BECN.) The congestion bits are provided for the benefit of the application (i.e., so it may take actions to prevent congestion problems from occurring.) The last two bytes of the packet (prior to the Ending Flag character) contain a Cyclic Redundancy Check (CRC). The remainder of the packet consists of the payload data.

Procedures - There are very few procedural aspects to the Frame Relay protocol. Packets are simply routed in the network via the use of DLCIs (Data Link Connection Identifiers), with each DLCI being configured to reference a specific destination system. Procedures are defined for congestion notification via the use of BECN and FECN bits. Any packet delivery problems have to be dealt with by upper layer protocols or user applications (which are implemented in the Customer Premises Equipment - CPE.)

Internet Protocol

Service – IP is a connectionless, unreliable Network/Internet layer protocol.

Purpose – IP provides an unreliable service to the Transport layer with no acknowledgements or guarantee of delivery. It does so by using the connectionless datagram service. It relies on the Transport layer (UDP/TCP) to define the reliability of the data traffic, sequencing, and any error correction. IP may loose packets and deliver them out of order. Thus, it does not perform sequencing, flow control, and has little use of the IP header checksum field. IP does however have options, addressing, and the capability for fragmentation and reassembly.

Packet – Version, IHL, TOS/IP Precedence, Total Length, Identification, Fragment Offset, Time to Live, Protocol, Header Checksum, Source Address, Destination Address, Options.

Procedures – IP can fragment packets that are too large for the underlying network.

IPSEC

Authenticates and encrypts each IP packet of the data stream. Protects data flow between a pair of hosts.

Packet – The AH Operates on top of IP using IP protocol number 51. Next Header, Payload Length, RESERVED, Security Parameters Index, Sequence Number, Authentication Data (necessary data to authenticate the packet). ESP is also a member. Security Parameters Index, Sequence Number, Payload Data, Padding, Authentication Data.

Procedures -

Internet Key Exchange (IKE) – sets up a security association by handling negotiation of protocols and algorithms and generates the encryption and authentication keys to be used

Authentication Header (AH) – Provides connectionless integrity and data origin authentication for IP datagrams to provide protection against replay attacks

Encapsulating Security Payloud (ESP) – provide confidentiality, data origin authentication, connectionless integrity

ARP: Address Resolution Protocol

Service – ARP provides automatic mapping from IP address to MAC address.

Purpose – Due to routing, it becomes necessary to find the physical interface address when given an IP address. This is because physical addresses only have relevance within local networks and an IP allows packets to be sent across networks. ARP is simply a means of asking for ownership of an IP address.

Packet – The ARP packet has the following fields: Ethernet destination address, Ethernet source address, frame type, hardware type, protocol type, hardware size, protocol size, operation type, sender Ethernet address, sender IP address, target Ethernet address, and target IP address. Notably, the operation type describes whether the packet is an ARP request, ARP reply, RARP request, or RARP reply. The Ethernet destination address, in an ARP request, is the broadcast address.

Procedures – To translate a Network Layer IP address to a Link Layer MAC address, ARP will first look at it’s ARP cache to determine if a translation already exists. Of course, this only takes place if the destination address belongs to the network of the current device. The ARP cache entries typically have some expiration time and thus if an entry is not found, ARP will broadcast an ARP request. This requests asks “if you have this IP address, please respond.” When the owner of the IP address receives the ARP request, it will respond with an ARP reply. Upon receiving the ARP reply, the data can then be added to the ARP cache and a Link Layer frame can be added with the correct physical destination address.

ICMP: Internet Control Messaging Protocol

Service – ICMP, supports IP at the Network layer. It helps communicate error and informational messages, whereas IP is relatively simple in nature and does not.

Purpose – Since IP is unreliable, connectionless, and un-acknowledged; ICMP was created to provide error reporting, diagnostics, and testing. Though, ICMP packets can be lost and discarded themselves.

Packet – ICMP messages transmitted within IP datagram with the following fields: type, code, checksum, and the contents. Type determines what kind of ICMP message it is and code helps specify the type even further. The checksum is calculated from the ICMP header and data.

Procedures – When a packet is inspected at the Network layer, it is possible that some condition may be acted upon and an ICMP message to be generated. This ICMP packet will be sent back to the sender. In a typical case, when a router receives an IP packet with a Time-to-live at zero, it will drop the packet and send back an ICMP message reporting “Time Exceeded.”

UDP: User Data Protocol

Service – UDP is an unreliable Transport layer protocol based similarly to the connectionless, unreliable IP protocol. It allows applications to access the IP with no bells and whistles.

Purpose – UDP provides a datagram-oriented Transport layer protocol. UDP provides no reliability, like IP. There is no guarantee that the datagrams will reach the destination. Thus, it is connectionless and data can be lost and transmission is unreliable. Also, is no flow control, congestion control, and segmentation. That said, UDP provides the capability for multiplexing and de-multiplexing through the use of port numbers.

Packet – Source port number, destination port number, UDP length, UDP checksum, and data. The port numbers are used for multiplexing and de-multiplexing; allowing many applications (and same applications) to use IP for network communication. For the checksum, a pseudo-header is generated with extra information for the calculation. These miscellaneous fields are source IP address, destination IP address, IP protocol field, and UDP length.

Procedures – Like IP, no prior connection is needed. Packets are just sent out with no need for acknowledgement. Thus no connection is needed to be initiated or disconnected. No error handling is used either.

TCP: Transmission Control Protocol

Service – TCP is a reliable Transport layer protocol and provides transport-layer addressing to allow multiple software applications to simultaneously use a single IP address. It allows a pair of devices to establish a virtual connection and then pass data bi-directionally.

Purpose –TCP provides a reliable, connection-oriented service to the application layer. Like UDP, TCP has multiplexing and de-multiplexing indentified with the use of port numbering. The checksum also provides some means error detection. The sequence number is used to identify each byte, providing data reliability. Flow Control uses the sliding-window algorithm to establish the connection and wait for acknowledgments. There is also congestion control that uses the Slow Start algorithm to prevent a device from overloading the network links.

Packet – Source Port, Destination Port, Sequence Number, Acknowledgement, (Offset/Reserved/ECN/ControlBits), Window, Checksum, Urgent Pointer, Options, Payload.

Procedures – TCP protocol operations may be divided into three phases. Connections must be properly established in a multi-step handshake process (connection establishment) before entering the data transfer phase. After data transmission is completed, the connection termination closes established virtual circuits and releases all allocated resources.

RTP: Real-time Transport Protocol

Service – RTP provides an unreliable but sequenced service to transmit data. It is unreliable for timeliness and sequenced to make sure data arrives in order. Data that is not in order is dropped.

Purpose – UDP and TCP do not meet the demands of Real-Time data. Data needs to arrive in order as fast as possible.

Packet – Version, Padding, Extension, CSRC Count, Marker, Payload Type (Type of Audo/Video and encryption), Sequence Number, Timestamp, SSRC (Synchronization Source), CSRC (Contribution Source).

Procedures – RTP provides end-to-end network transport functions suitable for applications transmitting real-time data, such as audio, video or simulation data, over multicast or unicast network services. RTP does not address resource reservation and does not guarantee quality-of-service for real-time services. The data transport is augmented by a control protocol (RTCP) to allow monitoring of the data delivery in a manner scalable to large multicast networks, and to provide minimal control and identification functionality. RTP and RTCP are designed to be independent of the underlying transport and network layers.

DNS: Domain Name System

Service – A service that uses a hierarchy of Name Servers to determine the IP Address for a human-readable URI.

Purpose – IP addresses are necessary to send data over the Internet. However it is more common for humans to remember readable names. Thus DNS provides a way to convert these names into IP addresses.

Packet – Indentification, QR, Opcode, (Many other single bit fields), Total Questions, Total Answer RRs (Resource Records), Total Authority RRs, Total Additional RRs, Questions, Answer RRs, Authority RRs, Additional RRs.

Procedures – Every machine connected to the Internet should have a local DNS server. Whenever someone attempts to hit some public domain, the request first heads to the local DNS. If the record is not cache, then it goes through a process of questioning the Root Name Servers, to a TLD Name Server, and eventually to the Name Server that contains the record which is being looked up.

QOS: Quality of Service

Service - To provide some guarantee of network performance for some given application.

Purpose – With Real-Time data, it may be necessary to allot some portion of the network to a particular application. Issues that occur in networks relate to: bandwidth, delay, jitter, error rate, etc.

Packet – Using the differentiated services code point (DSCP) markings in IP, DiffServ can indicate:

• Codepoint = 000000 Best effort (Standard Packet)

• Codepoint = 101110 Expedited Forwarding (EF) – strict low latency queue

Procedure – A few things can be done to provide QoS. One method includes increasing bandwidth. However, it is common to mark the packet using some specific criteria (DiffServ). Then each router will examine the packet to determine how to handle it. In this case, all routers in a network with QoS must be using DiffServ for this to work.

SMTP: Simple Mail Transfer Protocol

Service – Provides a text-based way to send electronic mail.

Purpose – SMTP is a relatively simple, text-based protocol, in which a mail sender communicates with a mail receiver by issuing simple command strings and supplying necessary data over a reliable ordered data stream channel, typically a Transmission Control Protocol (TCP) connection.

Packet – SMTP uses a series of commands. HELO, MAIL FROM, RCPT TO, DATA (headers and body), QUIT.

Procedure – After the message sender (SMTP client) establishes a reliable communications channel to the message receiver (SMTP server), the session is opened with a greeting by the server, usually containing its fully qualified domain name, in this case smtp.example.com. The client initiates its dialog by responding with a HELO command identifying itself in the command's parameter. With the rest of the commands, the sender can construct an e-mail message to store on the recipients mail server.

SIP: Session Initiation Protocol

Service - SIP is primarily used in setting up and tearing down voice or video calls. It has also found applications in messaging applications, such as instant messaging, and event subscription and notification.

Purpose - A motivating goal for SIP was to provide a signaling and call setup protocol for IP-based communications that can support a superset of the call processing functions and features present in the public switched telephone network (PSTN). SIP by itself does not define these features; rather, its focus is call-setup and signaling. However, it was designed to enable the construction of functionalities of network elements designated proxy servers and user agents. These are features that permit familiar telephone-like operations: dialing a number, causing a phone to ring, hearing ringback tones or a busy signal.

Packet - It is a text-based protocol, incorporating many elements of the Hypertext Transfer Protocol (HTTP) and the Simple Mail Transfer Protocol (SMTP), allowing for direct inspection by administrators. Commands include:

• REGISTER: Notify current IP address and the URLs to receive calls.

• INVITE: Used to establish a media session between user agents.

• ACK: Confirms reliable message exchanges.

• CANCEL: Terminates a pending request.

• BYE: Terminates a session between two users in a conference.

• OPTIONS: Requests information about the capabilities of a caller

Procedure - SIP employs design elements similar to HTTP-like request/response transaction model. Each transaction consists of a client request that invokes a particular method or function on the server and at least one response. SIP reuses most of the header fields, encoding rules and status codes of HTTP, providing a readable text-based format.

SIP typically relies on a Proxy server to help establish a connection with a remote user. A proxy server "is an intermediary entity that acts as both a server and a client for the purpose of making requests on behalf of other clients. A proxy server primarily plays the role of routing, which means its job is to ensure that a request is sent to another entity "closer" to the targeted user. Proxies are also useful for enforcing policy (for example, making sure a user is allowed to make a call). A proxy interprets, and, if necessary, rewrites specific parts of a request message before forwarding it." "A registrar is a server that accepts REGISTER requests and places the information it receives in those requests into the location service for the domain it handles." "A redirect server is a user agent server that generates 3xx responses to requests it receives, directing the client to contact an alternate set of URIs. The redirect server allows SIP Proxy Servers to direct SIP session invitations to external domains."

Questions from Review

T/F The IETF runs the Internet and its networks.

This is not quite a T/F question. Yes and No. The IETF produces technical documents that influence how people design, use, and manage the Internet. They do not however run the Internet. Many parties are involved in the distinction of managing the Internet.

Describe the 7-Layer OSI Model:

Layer 1 is the Physical Layer. At this layer, data is physically moved across a network encoded as electronic signals. Here the specifications for the hardware, encoding/decoding, signaling, and transmission/reception are defined.

Layer 2 is the Data Link Layer that is responsible for data that is transmitted between local devices. Error detection and error handling, logical link control (LLC), media access control (MAC), and addressing are important here. LLC allows this layer to abstract the defining physical network below it. MAC provides the capability for multiple machines to share a single resource. Additionally, MAC addresses are assigned as globally unique 48-bit numbers.

Layer 3 is the Network Layer which defines network boundaries and how they can be interconnected. The key protocol at this layer is the Internet Protocol (IP), commonly referred to as the backbone of the Internet. Important services at this layer are IP addressing, fragmentation and reassembly, error handling, and routing. The IP address differs from the MAC address and is independent of hardware. However, it must be unique at the network level and has two important parts: the network id and the host id. Fragmentation and reassembly allows this layer to split up packets that are too large for the link layer. Also, routing, determining where and how to send incoming packets, occurs at this level by inspecting the IP address.

Layer 4 is the Transport Layer. TCP and UDP are the main protocols that operate at this layer. Connection-oriented and connectionless services are offered in addition to keeping track of the connections software programs are using through ports. Like the network layer, data can be fragmented here through the process of segmentation. Moreover, important features include flow control, congestion control, and multiplexing and de-multiplexing.

Layer 5 is the Session Layer. Its purpose is to establish and control sessions between software.

Layer 6 is the Presentation Layer provides the capability to translate, compress, and encrypt software data.

Layer 7, the Application Layer, makes use of all layers below it and provides the capabilities that a user or system need on the network. There are many protocols that exist at this layer (FTP, HTTP, DHCP, NNTP, IRC, etc).

Describe the 5-Layer TCP/IP Model:

Similar to the OSI model, we have the Physical, Link (network interface), Network (Internet), Transport, and Application layers. From the bottom up:

The Physical Layer is responsible for transmitting the data over the network encoded as electronic signals.

The Link Layer handles the communication of data among local networks. Ethernet and the 802 protocols are commonly used at this layer.

At the Network Layer we have the IP protocol as well as ICMP, among others. This layer is responsible for routing and defining network boundaries.

The Transport Layer helps manage data communication across networks. It can do so with the TCP (reliable) and UDP (unreliable) protocols.

The Application Layer includes many application protocols that allow users and systems to use the network as a resource.

Describe how routers manage the water sprinklers at Fenway Park.

Personally, I do not know much about the sprinkler system at the Stadiums around the US. In this day and age though, I could imagine there being some central system that controls many of the day-to-day operations. One such program may be in charge of running the sprinkler system for a set period of time.

For the sake of the question, let this system be placed in some control room that’s off limits to most employees. There were concerns that the machine running the software could be tampered with so management wanted it to be locked away safely in some server room.

However, the job of the groundskeeper is to occasionally access this software to ensure that the field is in exceptional playing condition. The network administrator allows him to access the software through a remote and water-proof laptop. This laptop can connect wirelessly to a private wireless connection within the stadium.

To access the program, the groundskeeper can use an Internet Browser to open up the link that displays the controls of the sprinkler system (with the correct credentials of course). Thus, when doing so, the laptop is communicating with some wireless router which then itself communicates to the network in which the server resides.

Describe how video traffic is carried on the Internet.

Like voice traffic, video places an emphasis on timeliness over reliability -- as a reliable service can introduce delay. Real-Time Transport Protocol (RTP), defined at the Application Layer, was introduced to help stream media over networks. Since reliability is not of the upmost important, an UDP/IP datagram is used with RTP. UDP, unlike TCP, is un-reliable and is not subjected to Flow and Congestion Control.

Thus, when video is streamed out to users, we assume the following: that the sender is capturing and compressing the data, and generating the RTP packets. The software will determine how many frames, at which rate, and what the size of the transfer will be. Larger frames will traverse several packets where smaller frames can be squeezed into a single RTP frame.

The client’s application will receive the data in some buffer, with the capability of reordering packets that arrive out of order. Depending on the application, some algorithm may be used to delay playback unless a reliable stream can be viewed. Based on the amount of packet loss and jitter (packets arriving at differing intervals), a steady and clear playback experience may be possible after waiting for enough data to arrive.

How are routers involved with Harvard’s parking meters?

With the new pay-station meters around the Harvard, it is very likely that the meters are connected to the Internet or some private network. I would assume that the meters within close proximity to each other, perhaps all in Cambridge, share a common local network. Let’s assume that they are connected in a very simple manner using Ethernet switches.

Now there are two main reasons I see these meters connected to the Internet. One is for credit card validation and the other is for remote administration and reporting. Let us also assume that the meters run some stripped down Operation System that allows it do to this. Hence, the local network home to the meters must contain some Network Router. The OS running on the meters must also be aware of this router, and the interface, which connects to this network, will be considered the gateway. This router will then be connected to some ISP (Internet Service Provider) through another interface to allow data to travel remotely.

Thus, when a user uses the pay-station to purchase a 2-hour parking receipt, the meter will read the user’s credit card information and validate it with some on-line service. To reach the on-line service, the meter send traffic through it’s local router, to reach the ISP’s router, and through some interworking until it reaches the destination. The service will respond and perhaps ask for some credentials or the credit information. In turn, the meter can send the data over and eventually expect some verification.

Additionally, it would be useful for the county/police station to be able to monitor the meters. Take for instance a case where a reckless driver crashes into a meter. The meter may be able to send out message that travels to the local police station’s command center. Without routing and being able to connect to the Internet, it would take much longer for such an example to be noticed.

What are some differences between H323 and SIP?

See http://www.packetizer.com/ipmc/h323_vs_sip/

The biggest difference that I can tell is that SIP is better suited for the Internet and Internet developers. H.323 is better suited for Telephony Companies where more control is necessary.

WATDR: The LA Times Menu

2009-11-11T13:15:00.000-08:00

During the summer, when the LA Times rolled out their redesign -- I kind of fell in love with the new menus. It works well and it's drop dead sexy.

I wanted to copy it and noticed that the LA Time's loads a slew of JS and CSS files. Ugh! Where to begin? I just want a pretty menu...

Instead of drilling through and finding the code I needed, I knew it would be faster to just create it ad hoc. Heck, it couldn't be that hard with our friend JQuery. Plus my version might be faster and lighter weight.

Cool, it wasn't too hard. It only took me an hour or two to get it in a state that I wanted. Now six months later, I thought I should blog and share it with you. My menu isn't a blatant copy. I kind of felt bad about stealing too much of the design but I wanted to emulate the feel. You can make it look almost identical with the right images and a good amount of patience.

http://donniedemuth.s3.amazonaws.com/MenuExample.html

If you're looking to do something similar, I made it really easy for you to figure out what's going on. Go ahead and click on the following link and feel free to copy the source code. It's so easy a caveman can [read it]. (After clicking on the link, right-click and view source)

Ask your “bestie” to help you style it and you'll have the prettiest menus on the web. If you liked it, please let me know!

WATDR: What on Earth is CNN doing?

2009-11-11T12:27:00.000-08:00

Who doesn't love all the new things CNN has been rolling out? In my opinion CNN is the front-runner in determining how we all will visualize the news in the future. Here's a quick post about some of the new things I've noticed in the past 24-hours.

NewsPulse

Right now it seems that the organization is making a strong push for the reader to use the NewsPulse feature. It's now integrated at the Story level (when you're reading a story its on your right-hand side)

In general, NewsPulse is a data-centric way to drill through stories. This makes sense since most organizations use databases to store their data over the old-way (circa 3-5 years ago) or just having a whole slew of HTML files.

Story Photo Slider
A great way to visualize what happened in a story is to be presented with it's associated images up close and personal. When I'm reading a story, I'd like to see the lead photo as large as possible and I'm glad CNN gets it. CNN goes above expectations by throwing in a sweet little slider for the reader to easily "watch" what happened.

It's a fantastic use of space. You can even hide the captions is if you want. And although I loathe sliders, this one works well. If I were calling the shots, I might ditch the slider and provide users with a link that displays all of the photos (and captions) at some high-resolution capacity. Kudos to Alan Taylor, at the Boston Globe, for bringing this into pop-culture with The Big Picture.

Story Highlights

It's kind of weird that I own the domain storyhighlights.com. But as you expect, story highlights are the cliff-notes version of a story. I want to see this everywhere. I tend to read the highlights first to determine whether or not I want to read the full story. It used to be at the top-right of the page and now it's pressed left.

Below it, on the left-hand, are Related Topics. Some sites call this tags or categories, but it's really nice to see that CNN limits how many related topics there are. The LA Times used to provide a ginourmous word-cloud which was a terrible idea. Example:

Well, it wasn't exactly like that but it could contain more words/tags than the story itself. Information overkill. I'm glad that I just recently noticed it was gone from the LA Times.

The Sliding Left Wall
Not all is super smooth in CNN land; though I'll probably change my opinion in the future. While your reading a story, the left-edge where words begin on a new line sometimes presses right.

I personally think that a little weird. You don't see that pattern in books or magazines that often when reading one body of text. I feel that it distracts the reader from reading a passage in it's entirety -- and heck doesn't it just look funny? I'm sure there's a purpose and maybe you can tell me why it makes sense. In my world, only the right-wall would collapse for external objects.

WATDR: Guardian.co.uk

2009-11-06T07:07:00.000-08:00

What are they doing right?
I decided to start a series of posts about the changes I'm seeing in the news industry. After years of providing stale online information, it seems this business is now in the business of trying to capture your attention. The newsroom is now asking themselves “how can we get people to come to our site” rather than relying on the belief that “people will come to us for the news”.

With growing competition and freely available news on the internet, it seems more apparent that online advertising is vital to the success of journalism and publishing. Besides excellent written content, usability and aesthetics are important for capturing this tech-savvy audience. In this world, you'll need water to survive.

Guardian.co.uk
Let's begin with the Guardian.

Unlike many news websites I've come across, the use of space is exceptional. I doubt information overload is a problem with the Guardian and their team has made several wise and risky decisions. Rather than putting everything imaginable on the homepage, your intuition can drive you to where you want to go. For a website that someone that may visit everyday, a smartly designed site with an increased click-count can be negligible. Overall the site seems to have a very inviting feel to it.

Homepage Layout

The homepage actually has a four-column layout. In the left-most column you have the headlines for today. I would be surprised if the top headline didn't appear at that top-left edge. From that edge, various links and images branch out across the page in nicely compartmentalized units. It's very easy to tell where one point of interest begins and ends on the page.

Caption Space Saver

I am not sure what to call this, but I like it a lot. To save space, captions are displayed as you hover over certain images (mainly in the middle-right column).

Simple Menus

The menu is minimalistic, behaving quite nicely. I was surprised to see that there were no hidden sub-menus eagerly waiting to appear. I kind of like that! But I'm sure there are management teams that disagree with this simplicity. Many whom I've come across want to throw everything into menus to keep the click count low. The balance between intuition and click-count will vary from team to team, or country to country in this case.

For those that insist that every page has 1 or 2 clicks to every other page, it may be worth it to consider a site-map drop-list. I feel that I don't see many of these and they work surprisingly well. On the Guardian.co.uk, this site-map always appears at the top-right corner of the page.

Accessibility

Also, at the top-left corner of the page are accessibility options. I like how it takes little thought of my own to find the mobile version or increase/decrease the font on the page (although I would use keyboard shortcuts for that). One thing that I am not a fan of is having the Log-in/Registration links on that side of the page. It's far more common for that to appear in the upper side corner of the page. I would have preferred that the Guardian.co.uk followed the majority here.

Extremely Easy RSS

Did you know that you can type “/rss” at the end of any URL to receive the RSS version of any page? Yes, you can get an RSS feed for nearly anything including stories and the homepage. What's amazing is that when you get the RSS for a story, you'll retrieve many of the associated stories and media files for it. For example, today's big story is about the shooting at fort hood: Fort Hood gunman Nidal Malik Hasan shouted 'Allahu Akbar' as he opened fire. Within this feed, we also get stories and links for:

I think that's pretty amazing that the Guardian is so open with providing this data.

Combined Topics

An interesting idea is to provide to users the ability to create sections. On the url you can combine multiple topics with the plus-sign... And sometimes it works and sometimes it doesn't! In the above picture, it works but doesn't return any results. Unfortunately I have a hard time getting results when I combine topics and it seems like topics must come from the same section. But I really, really like the idea and I recently added similar functionality to SignOnSanDiego based on tags (though I don't think stories are actually being tagged correctly since I left).

Btw, you can get an RSS feed of combined topics just by adding /rss at the end. Pretty nifty!

Story Detail Layout

Stories look beautiful at the Guardian.co.uk. Notice how the lead photo is enlarged and centered. I expect that and not enough news sites do it. And below the lead photo rests the story's text with ZERO nudging. That is, there are not parts of the story pressed in left or right. I like that because my eyes are trained to read in that manner. Oftentimes you'll have pictures or additional info that that wrap the text around it. Example (from another site):

When this happens, your mind momentarily breaks a pattern. It was used to looking exactly at a certain edge when reading the next line. Sure, it's not a big deal but I think it matters more than most people think. People should be able to read as easily as possible. I want to know, actually not even think about, where I should be looking. The story's text should flow as illustrated by the Guardian.co.uk.

Colorful Topics/Categories/Highlights

I enjoy the presence of the colorful topics at the top left of the story. I think it looks great and clicking on certain topics helps you drill down into other sections. It behaves like a sub-menu that's tied into some folksonomy architecture.

Also notice how directly below the headline are story highlights. I'm not sure whether or not CNN or the Guardian invented this, but it definitely helps me digest stories within seconds. In a few years, I expect every news organization to provide a similar bulleted summary.

Which paper should I look at next? I'm deciding between The Houston Chronicle or The Detroit Free Press, mainly because I like their domains.

Redesigning the News

2009-11-01T18:48:00.000-08:00

Yesterday, the Newspaper that I used to work for redesigned their homepage. I was actually involved in some capacity as a consultant. My hands only touched back-end stuff like the Associated Press integration and tagging. This meant I had no involvement in the front-end design but I wish I did because in another life I would be a User Interface Designer.

Unfortunately, I felt that our website SignOnSanDiego initially missed the mark. The launch was riddled with bugs and the system wasn't playing friendly with FireFox. Fortunately the team solved many of the problems relatively quickly.

SignOnSanDiego - Homepage

The homepage looks a little more modern. Off the bat, we have larger pictures, a slider, and a new logo. More media, pictures and video, is the what the news industry thinks people want. But by presenting the tops stories in a slider, I'm not a fan. Sliders are distracting and annoying to say the least. I try to read the story's tease and BAM it changes on me.

You'll probably notice the logo. Is it in the archetype of a reputable news organization? I'm not sure, but you could probably slap it on a skate or surf board and still look cool.

SignOnSanDiego - Story

I don't want to comment on the story page because it must be a work in progress. Why is the lead photo on the left and pushing the story's body to the right? Yuck. I would either center blow up the lead photo or have it float to the right.

Overall, the feel is amateurish. But knowing the deadlines at their office I know they probably have more on their plate than they can handle! I hope to see a sleeker site in a few weeks.

The News and Homepage Redesigns
Who started this trend of homepage redesigns? Within the last three months I've noticed that almost everyone has jumped on the makeover mo-ped. If I were a gambling man, I would pin it on the Los Angeles Times. I noticed their change first. Let's look at it and some of the homepages for several of the larger news sources. I'm not yet a fan and hope to see some improvements here.

Los Angeles Times

I thought the old Los Angeles Times website was amazing. The light blue, white, and beige color scheme was beautiful. It already seemed modern and up-to-date. I was completely shocked to see it change. Though I was even more surprised to see it change for the better.

What's the new color scheme? It's black and white. We're talking about the news industry here. How did they pull off black and white and make it look hip and edgy whilst appealing to the young and old? As Paris would say “That's hot.”

You have large text, 16:9 photos, few ads, and a menu that works really -- really well. Notice how the menu doesn't overlay anything else. It belongs within two horizontal bars across the top of the page. Excellent! I commend you Mr. LA Times designer.

CNN

CNN's homepage looks like the news. There redesign seems to be focused more on usability rather than piling on the bells-and-whistles. And it works surprisingly well. What I notice most is that the Latest News links are almost immediately at your left-hand side. Great! It's the first place people look and they're probably looking for that.

The Washington Post

I'm really surprised here. In my opinion, The Washington Post has always been solid. I'm not sure why the new/local homepage looks the way it does. Is the color scheme vomit-green and grey?

Want to see some nasty menus?

And is it just me or does the Washington Post hate Opera? It won't even load in it.

The Guardian

The Guardian is always evolving. I swear it looks different every few months. Overall, I find the site very clean and easy to navigate. Not only is it one of the oldest news organizations in the world, they are outright trendy. Props to the Guardian for using some random colors, like two shades of fushia.

The New York Post

The New York Post has the best search menu bar ever. It's always where you'd expect it to be. Though, I think I have a love+hate relationship with The Post. At times, I think the site looks amazing. At others, it is FAR too cluttered.

The Post wins my heart with it's latest stories for each of the big NY neighborhoods, under NYC Now. It's perfectly aligned on the left edge of the page. But it's not all good for the Post. The menus, I'm sure with good intentions, is far too fanciful. Simply put, it does too much for a menu. I'm certain it would alienate the older crowd and I would expect the designers to go back to simpler system in the future.

The New York Times

I'm certain we'll see a redesign from The New York Times by the end of the year. Bank on it. One thing I hope they don't lose is the left-handed menu. How many times have I mentioned the phrase left-handed in this posting? Plenty! Because left-handing things are important to the user. It's where your eyes are trained to look. The NYT is the only news source that I know of that still denies a top horizontal menu. Ain't that strange... and oddly smart of them.

The Huffington Post

Okay you got me. The Huffington Post is a blog, not a news outlet. But did you catch the lead story though? Oh what fun it must be to be the Editor at The Huff. Seriously, I think it's a little too big.

Homepage Rankings
From sexiest to least, I'd say:

The Los Angeles Times
The Guardian
The New York Post
CNN
The New York Times
SignOnSanDiego/The Union-Tribune
The Huffington Post
The Washington Post

I'll come up with a real rankings system in a the week or so.

See Don Walk

2009-11-01T09:54:00.000-08:00

Sometimes walking is faster than taking public transportation.

It's not that far, only 2 miles!

And Lower Allston isn't that ghetto at all.

Sooner or later you'll pass Harvard Stadium.

On the other side of the stadium is Harvard Business School.

The Business School's Library is really pretty.

After you cross the river, you'll enter Cambridge.

It's an urban neighborhood.

I still haven't seen my bus. I win.

Harvard Yard has been around since the 1630s.

Tourists love this place. Say “hi” to John Harvard.

There are a ton of libraries.

My computer networks section is in one of the big red buildings.

I think the kids get to stay in here.

Mother likes churches so here here's one.

But I like the architecture on this one much better.

Lawyers apply to law school here.

And study at the Law Library.

I usually end up studying there with a friend.

The computer science building doesn't have that majestic feeling.

I usually sit in the front-left row.

But I also have a class and a few sections in this ugly building.

But luckily it is right next to the computer lab.

Welcome to what I see 50% of the week.

Sometimes I sneak into a journalism class in this room.

All this walking makes me hungry.

The Fall 2009 Harvard Extension Bi-Semester Review!

2009-10-25T09:02:00.000-07:00

I highly recommend The Midnight Meat Train. It's a horror flick that gives you everything that you would expect in that genre, including a huh(?)-ending.

Now let's get onto the fun stuff. If you've been following my blog you know that I'm currently taking three classes: CSCI-E 131b, CSCI-E 168, and CSCI-E 207. It's highly likely that if you're thinking about the ALM program that you will be taking one of these classes. Since we are at the halfway point, let me share my grades and a few thoughts.

Computer Networks and Internet Protocols
Grades
So far I've turned in two problem sets. They both took a bit of work and I'd like to share some of my research in later blogs. If I had to guess how long each took, I'd say about a good 10 hours of writing each. In the end they both consisted of 10 pages including diagrams.

HW 1 : 19/19
HW 2: Not yet graded
HW 3: Just started, seems pretty difficult

Flow
I find the flow of this class right on. It's not too difficult or easy for me, but I don't have a ton of computer network experience. Learning about protocols is kind of what you'd expect it to be. It can tend to be really dry. Fortunately, Professor Evenchik and the Teaching Fellow "Joe" are pretty awesome when it comes to explaining the material. I find it interesting to talk about and difficult to read about.

Should You Take It
In a single word, yes. Not being much of a network/systems guy myself I feel much more comfortable understanding how the Internet works. It's been enjoyable. Heck, I even know the implementation details behind traceroute.

Web-based Software with Ruby on Rails
Grades
We've had three programming assignments. It's hard to describe how difficult or easy they were. To be honest, at first they were incredible hard. For some reason it's hard for me to decipher the assignment objectives (example: Assignment 2). But after spending a few hours understanding what you're supposed to do, it's not that bad.

Assignment 1: Ruby One Liners: 97/100
This was actually a fun assignment. I spent a weekend working on this for 8 or so hours the first day it came out. If I waited long enough, Professor Norman will cover each answer during class. Thus, the longer you wait the easier the assignment will be.
Assignment 2: Classes and Methods: Not yet graded
Again, this was fun. We build upon a game that Professor Norman and his staff created. It's a command prompt game that's similar to pong and the game of life. Basically all you need to do is to implement the missing methods and build a few objects of your own. I thought it would be cute to add blood-thirsty zombies and vampire slayers. I spent 10 hours on this.
Assignment 3: Active Record: Not yet graded
You're assigned to save the state of your game in the database and restore it. A good two hours or so got the job done.

Flow
I really, really like John Norman and the Teaching Fellow "Keith." Really cool and smart guys. For the most part, the flow is okay. I've never taken a class like this so its hard to say. I think I had different expectations. The class could be more accelerated if there was more trust that the students were capable of learning on their own. Maybe that's a good thing as everything in class is served on a silver platter. You'll definitely learn in this class. But thank god I'm swamped with my other classes and part-time jobs.

Should You Take It
If you really want to learn Ruby on Rails, then sure. People recommend this class all the time. Plus, the documentation and tutorials on R/o/R are so good it's very easy to teach yourself. Being more comfortable with Ruby, I respect it now as a language. Rails on the other hand... It's not my favorite framework. I'd like to learn more about the other Ruby frameworks. I'm bitter that Python and Django is not getting the hype that it deserves.

Formal Systems and Computational Theory
Grades
Five problem sets and a mid-term. That's almost one problem set every week. How long does a problem set take you ask? About twenty hours.

Working on them is probably the hardest thing I've ever had to do in my life. Just ask anyone else in the class. They agree. One out of every one students will confirm that this is tough-cookies. The assignments are posted on the web, take a look.

PS 0: Does not count
PS 1: 61/62, average 50.69
PS 2: 52.5/56, average 41.63
PS 3: 61.95/64, average 50.55
PS 4: Not yet graded
Sadly, I totally muffed one of the problems w/ about a page and a half of work. Gosh, it was a 15 pointer also.
Midterm: Not yet graded
The midterm wasn't too difficult. I spaced out the first five minutes and I think I was way too anxious. I got really nervous and just read every problem about 3 times. Then I tried to calm myself down and breathe slowly. Out of seven or eight questions, I was only kind of confused about one of the problems. The problem was a memory based one, where you either knew the definition or not. I'm hoping that I beat the average here.
PS 5: Just started!

Flow
The class consumes you. It gnaws on your toes and spits out your nails. Be prepared to work and you must devote a large block of time to this class. You can only take this course and feel overwhelmed. Luckily, I only work part-time but combining this with two other classes is very difficult. I've stayed up til 4~5 A.M. several times this semester already.

I HATED this class around Week 4. But since this class takes so much time away from everything you can possibly think of -- the class becomes you. And you become a theoretician. And you realize that something is missing when you're not working on problem sets.

Should You Take It
One should ask oneself if he/she is masochistic.

Ghetto-Fab Teaching with Technology

2009-10-14T08:48:00.000-07:00

ghetto-fab [urban-dictionary]: in my use I mean using something for free when there are paid and probably better means

I loved it when Ali-G said things like, "Tek-mol-lo-gy... It's happening." And right he is, it is happening!

Thus, how can we use modern (say Web 2.0) technology to teach or assist in teaching? As a remote instructor at UCSD-Extension, all I really have is BlackBoard and PowerPoint with voice-over. So far it has worked out really well. But it does feel a bit old and really does feel like remote-learning. Hint: it's not a warm and fuzzy feeling.

This upcoming semester, I am going to try out a few new technologies. If anyone has tried one before, please let me know! [note: this post will grow]

Justin.TV
I'm also attending school and several of my teaching-fellows use a tool named Elluminate. It's more or less a chat room with a shared MS-Paint window. Elluminate is a useful product but I almost never see the drawing window being used. Most of the conversation takes place in the chat room where almost any chat software would suffice.

For a while, I thought to myself "Hey why can't I teach with Elluminate also?!" But then I realized, heck why bother when I could be streaming live. Justin.tv is a nifty website that's been around for a few years that allows users to stream themselves. Perhaps I could hold things up to the camera and at the very least let my students feel like a human-being is teaching the course.

What's nice is that there's also a chat room feature with all of the common features like private messaging. What's even better is that there's desktop streaming. Yup, I'll be able to code and allow me students to see it. Doesn't it seem like a great way to show examples?

Google Wave
I'm sure there will be a ton of people trying out Google Wave for education this upcoming year. It seems like it would be a good tool... I'm still not drinking the wave-water though... I'll need to think on this. TBC...

How To Do Not Poorly Through Week 6

2009-10-10T08:40:00.000-07:00

Update: Oh cool look at my G-talk status about this post: "How To Do Not Poo".

~*~*~*~*~*~*~*~*~*

I'd like to start off with a short advertisement: If you want to be organized -- hell I'm a left-handed, left-thinking spaghetti-code mess -- invest in Unikeep binders. They are wicked cool. And somehow keeping my life right.

Week 6, And Things Look Better
I am not 100% sure how well I'm doing in school. Heck, I'm always worried and second guessing myself. However, I think it's now safe to assume that I am not doing poorly! Here's why:

CSCI-E 131b (Computer Networks)
I took the time to complete the homework a week and a half in advance. Additionally, I've somehow forced myself to read all of the reading material even though most of it goes right over my head. As you could imagine, it is impossible to tell if your peers have done the same.

This Wednesday, our professor took the realms of section lead over the awesome "Joe-Leader." What's funny about this is that instead of talking about the material in a practical light, Professor Evenchik interrogated the room! He asked everyone for a set of questions. And waited until you returned a response.

I had plenty of questions but surprisingly there weren't many others. Additionally, it seemed that, in this small section, hardly anyone else started the homework. Now I don't think I'm necessarily ahead of anyone else, but it's nice to know that I was somewhat prepared for his surprise.

Hints (so you can do well too):
Start the homework early. Skim the reading material. Go to the computer lab at 53 Church Street to print out the material for free. Read Wikipedia for some of the general topics. Wikipedia will spell out the concepts in layman's terms. I know it's easier said than done, but prepare a few questions for section or class.

CSCI-E 168 (Ruby and Rails)
Ruby, Rails, easy stuff. I get it. It seems that many other people don't because they keep questioning the "philosophy."

Whyyyyyy can you do that?
Jaaaaavaaaaa is strict and therefore better!
Caaaaaaaan't that lead to bad code?

Come on people. Haven't you used an api before? Learn how to use things and appreciate the features that it has to offer rather than finding every single reason it's not the same as language X. It's Ruby. Did you do no research on the topic before joined the class?

Sorry, the class is a little aggravating. I love the lectures. The instructor and section leader are way cool. But I do wish that the students in the class were a little more progressive and modern -- when it comes to technology.

Hints (so you can do well too):
Don't question everything. Try to see the advantage of some of the features that Ruby provides. There are pros and cons for everything, convince yourself that the pros outweigh the cons. Think modern. Be 2009 -- 2010. Stop living in the 80s -- 70s -- 60s -- 50s -- 40s -- 30s.

CSCI-E 207 (Formal Systems + Computation)
Okay, my nightmare. My thorny devil. The bane of my existence, CSCI-E 207. How I love you and how I loathe you.

But how I think I'm getting to know you.

Now I'm constantly behind on lectures. And it's very difficult to keep up when the material moves so fast. Interesting enough, for the material that I make it to I think I'm doing okay. I was actually able to complete most of Problem Set 3 without the headaches of past. Amazingly, I am also feeling more comfortable around this material that I can actually communicate about it. Heck I even "aced" one of the sets (heh, after extra credit).

*Oh Donnie! The sky's the limit! Things are looking brighter son!*

If you take this class, YOU BETTER GO TO SECTION and you better hope that Brian's teaching it. Otherwise, you can go straight to www.harvardfml.com.

Hints (so you can do well too):
Go to section. Cry here and there. Keep a hearty supply of Kleenex. Pour any whiskey down the sink. Do not walk on train-tracks or in the middle of the street. Do not buy any cyanide capsules, etc. Do not talk about fight-club. Serve Project Mayhem.

Ethics and Journalism E-120

2009-10-07T15:10:00.000-07:00

I had the chance to sit in an Ethics and Journalism session at Harvard Extension. Actually, the class just happened to take place around me -- while I just sat. I have never taken a real writing class before so I was really intrigued with the presenter, material, and the students. Everyone was engaged and participated. Just about everything I'm not used to seeing in a Computer Science class. *Sigh*

I believe the speaker was Allan R. Ryan and he had the splitting image of Anthony Hopkins. I could probably listen to him all day as he had such a commanding and majestic (is that odd?) voice. On this particular day, they were talking about an article by Nicolas Kristof at the New York Times. The article, Media's Balancing Act, discusses when is it appropriate to publish a story without concrete evidence. Topics included a case with a priest with molestation allegations and a new lead in the Jon Bonnet Ramsey case. The response by the class was fascinating and I wish I was part of it.

Some of the statements overheard were:

Get the facts right before you run with a story!
What if the suspect presents "emotional blackmail," where you will feel responsible for the harm that may come from the story.
Do you take the risk of publishing a non-story if it will prevent harm to others?

Getting There, But It's Not Looking Good

2009-10-02T13:32:00.001-07:00

CSCI E-207 Week 5 Update

Have I ever mentioned how challenging CSCI E-207/121 is? It's definitely not for your normal-minded human. Ha, I thought it would be "cool" to actually learn about the theory of computers. Heck! Why wouldn't it be cool? Unfortunately, theory and theoreticians: "are who we thought they were."

Basically, a prerequisite for this class is a certification in genius. And an admiration for "this guy."

If he seems like someone that might stomp on your dreams, then maybe a theoretical class ain't for you. Jokes aside, this material isn't for the weak at heart. If you can't figure everything out on your own then *honestly* you're going to have a lot of trouble. Theory has always been one of those things that those that get it, get it, and those that don't, don't. No one is going to hold your hand and help you understand.

P.S. I'm just bitter that I'm not a genius. =) Oh and I dunno, my neural-pathway-thinga-ma-bobs are leading to dead ends. Grr...

CSCI 121 and 207 HQ Lectures on Miro

2009-10-01T08:44:00.000-07:00

The Videos That You Paid For
I don't know if anyone else is unsatisfied with the live video feeds during a Harvard Extension lecture. Personally, I would like a better video player. A High-Quality live stream similar to Justin.tv would be grand! I know Harvard should have the bandwidth and technology to provide this, but obviously I'm blogging for a reason.

Right now I am currently stuck with a 320x240 feed where the professor is drawing figures and hieroglyphic symbols on the chalk board and it's not always clear what I'm looking at. Additionally, it's kind of sad that you can't really pause-and-resume to take notes and try to decipher whats taking place in the feed. Oh, and don't even think about restarting the stream or trying to start the video up at a particular frame. Tracking isn't enabled either. This is pretty much what you get:

Though more features are available when you're watching the replay of the video. And it really isn't all that bad. But I'm sure if you are going through the bother of reading this, the next part will brighten your day.

The Same Videos For Free
So I'm one of those stupid guys who fails to bookmark really important sites -- like the CSCI 207 homepage. That means, every time I need to find the link for the video feed I have to first search for the 207 homepage... Yes, it's very annoying and I do this twice a week.

Fortunately, something good happened in my life the other day. When searching for the homepage using the terms "CSCI 207 Harvard" I somehow clicked on a link that had lectures from 2007. It's neat because it uses the Quicktime player and you can save the lecture to your disk.

In addition, if you want next's week lecture or any lecture for that matter it's all there (considering that Theory lectures do not change often semester to semester). I haven't looked ahead yet but I'm loving the ability to have some tracking and control over the video player. If you plan on taking this class I highly recommend watching one of these lectures to get a good feel of the class. Surprisingly, I did not find the videos posted on the other popular education sites like Academic Earth.

If you're currently taking this class, try the Miro video to help supplement your virtual classroom experience. The awesomely-cool folks could probably put this video on their iPod to learn about theory on the go.

The Havard Computer Lab, for Extension Students

2009-09-28T12:23:00.000-07:00

I finally decided to check out the computer lab that's available to us extension students. It is located at 53 Church St, maybe a block away from 1 Story St (a common place for Computer Science extension courses). In the picture below, I highlighted the computer facility in red and 1 Story St in purple.

Gaining Access
From the street, you'll see a cute little garden. Entering it you'll most likely see a sign above a heavy glass door that reads "Harvard Computer Center." Unfortunately, you can not waltz right in. You'll need an access card.

Luckily, to get the card all you have to do is push the call button (it's on a button panel to the left of the door). Ask the receptionist that you would like to get access to the lab. She'll ring you in. And all you need to prove your identification is some picture ID. From that they'll be able to determine whether or not you're a current Harvard Extension student.

What's Cool About It
Okay. I don't know where to begin. There's a room with 23+ Apple Cinema displays and souped up MacPros. There are also MacBook Pros and a Windows lab for Windowey-type of people. Printing course documents seems to be free and uber fast. And it's all very quiet and hardly full.

Though saving the best for last, there are two sets of bathrooms. The "Head"-to-'person-head' is excellent!

CSCI E-207 Week 4 Update

2009-09-25T12:43:00.000-07:00

I am going to need a LOT of luck.

Harvard Extension
CSCI E-207: Introduction to Computational Theory
The first homework was very difficult. The second homework is just as hard, though starting to make more sense. I really want to understand this material but I feel that I will never be intelligent enough for it to click. Actually I may be scaring myself off from doing well. At times I feel like I know what I'm doing -- At times isn't often.

Besides some of the other recommended books, I found that Daniel J. Velleman's How To Prove It: A Structured Approach is really helping me understand proofs. Oh yeah, you're going to need additional books. The one required book is hardly helps someone like me (a nitwit) understand what the problems in the homework mean.

Class
For extension students, lectures or stream or recorded. For the most part, I kind of like the lectures. The material is fairly general and I never find it completely over my head. However, I am not quite sure if watching the lecture will help you do well in the class. The homework is much harder than anything discussed in class and I suppose there's a valid reason for that. I just wish that after reading a chapter and watching lecture, I would know how to solve a problem. And I find myself far from that aspiration.

*Ding-ding* But the lecture itself is intended for real Harvard students. I guess they really are that much smarter than, me (where I do not know how I relate normal folk and I don't want to insult you).

Section
Our section leader, Brian, is probably the only reason I am still in this course. The guy is total genius and during the last section I probably hammered him with some of the most idiot questions he's ever heard. He handled them properly and helped clean up the mess in my mind. What I did find strange is that the other students in the class seem to be doing pretty well. I was hoping that I would find myself in a pool of the lost and confused. Maybe it's just me.

If I bomb this class, and good god I hope I don't, I know that I'll understand theory at some elementary level at least. Brian's section will save your butt.

Pixies
Gouge Away
I'm sticking with it but I bet I'm going to suffer...

CSCI E-168 Week 4 Update

2009-09-25T12:01:00.000-07:00

So far the R&R class has been pretty easy for me. Note: I was a little familiar with the language about four-five years ago so I have a bit of a leg up in early in the semester. I am learning a whole bunch though and I'm loving it!

Harvard Extension
CSCI E-168: Web Applications with Ruby on Rails
The first homework involved writing Ruby one-liners. That means stringing along blocks and methods to create neato functionality on a single line. I was able to accomplish this task in a few days fairly early. Since then, we've stumbled on better ways to writing such tasks and we're starting to get into meta-programming. Fun!

Class
The professor, John, is really-really good. I believe he said he was not a computer science major in undergrad and it shows. His presentation skills are slick and I'm sure he'll be a big name in Ruby if he isn't already. Like my Computer Networks class, I haven't tried listening to the stream of the class but I bet they are awesome.

In general, the class correlates strongly to the reading assignments and the homework. If you find the homework to be hard, just attend class and section and it should make more and more sense. I have a feeling that John wants everyone to actually do well and it would be pretty difficult not to.

Section
The section for this class ended up on Thursday afternoon around 5:30 PM. Thus, our section tends to be pretty small as most people can not make this time. Our section leader, Keith, is a hip (cool, whatever) guy who kind of reminds me of myself. In fact, his teaching style is a lot like mine which pretty much is "don't sweat the small stuff."

I can tell that some of the more mature students in the class want to sweat the small stuff but hopefully they will come to terms that convention is more important... than let's say how fast bing-bong and doo-wap runs.

Unfortunately, I haven't asked many questions in this section. To be honest, I feel pretty comfortable and I feel pretty happy that I "get it." I would like to get my name out there eventually. I think the leader knows who I am though and he seems like a guy I'd like to get a beer with.

Pixies
In Heaven
Whelps, I think I'm doing well!

CSCI E-131b Week 4 Update

2009-09-25T11:36:00.000-07:00

We are already in week four of the semester, yikes! And I am sorry that I have not been able to blog as often as I liked.

Instead of a detail summary of the material that I am going over, I will talk about the class in general and how I feel about the program. I would really like to make this blog more technical, but the courses are actually kicking my butt! That might help me actually get around to writing and inform those curious about taking certain classes.

Harvard Extension
CSCI E-131b: Computer Networks and Internet Protocols
The first homework assignment was pretty difficult. I probably wrote five pages at size eleven Arial font. However, after going to section the other day I realized that I made the assignment much hard than it should have been.

Basically the homework asked for, so it seemed, detailed responses to fairly technical questions. For instance, draw a sliding window diagram for XYZ protocol. Although I knew what a sliding window looked like, I didn't quite know how to show it in depth with the XYZ protocol. Fortunately, our section leader told us that the responses did not have to be that technical at all but show that we understood the general idea.

Class
Attending class still seems to be entertaining. I think that our professor, Len, is still an excellent presenter and I am happy to be there in person every week. I have a habit of sitting in the front row but I suspect everyone in the back will have no problem listening to his lecture. Unfortunately, I haven't tried listening to the stream of the class so I can't really judge how the recorded sessions are.

In general, the class correlates strongly to the reading assignments and the homework.

Section
So far we have only had two sections so far. They happen to be on Thursdays around 6:30 PM in a cozy, old-school (ha!) room near Harvard Yard. The section leader, Joe, is an amazing guy himself and could probably easily teach the class. What I like about him is that he really tries to help us out, no matter how idiotic the questions may sound.

The amount of students that show up has been pretty small so far. About 10 students the first section and maybe 5 the next. I recently decided to speak up more often in section because I really want to do well and no one seemed to mind. Unfortunately, I probably asked about 9 of the 10 questions that were asked during the hour. Cool-guy-Joe didn't seem to mind because it keep him busy scribbling on the blackboard. I really enjoy the section and I find that it adds value to the overall course.

Right now we're learning about the Ethernet and LANs. A few questions that I asked were:

Q: Does it ever make sense to use a hub over a switch if they were the same price?
A: No, in fact you it would be difficult to buy a hub these days.
Q: Does the switch-table keep track of other switches on the network?
A: No, switches do not have Mac Addresses! But routers do.
Q: I find the IETF mailing list very intimidating and I do not see how I could ever contribute. Is this normal?
A: YUP! Those guys are geniuses.

Pixies
Hey
Networking... It's just one of those love/hate relationships.

CSCI E-207 Introduction to Formal Systems and Computation [Intro and CH. 1]

2009-09-09T18:50:00.000-07:00

Theory scares me. It is probably because I have tried to avoid it my entire life. Maybe it is because I consider myself a practical person, not a theoretical person. But to be honest that is not a fair argument. As a computer scientist, I need to become familiar in this area and it is the primary reason I am taking this course, CSCI E-207.

As of right now I do not understand much about computational theory other than a bit about big-O-notation. And from several of the lectures and the reading assignments I have already looked at, there is a lot more that I more than I need to know.

Instead of writing a summary, I think for this class I am just going to blog my thoughts for the time being. I feel pretty uncomfortable about the topic to actually be writing about it.

The Book
I think I enjoy the book so far. It's "Introduction to the Theory of Computation" by Michael Sipser. The author explains the material well but for some (including me) it is a bit too brief. I am in the process in looking for a companion book to cover this topic.

Lecture Observations
The professor is actually a little entertaining to listen to and he speaks very well and clearly. The power-point slides are very useful and the professor hardly ever reads them word to word. Actually, there are many times where he will break from the slides to question the class or illustrate a topic on the blackboard.

Current Assignments
The first assignment seems is quite challenging. There are a few easy questions (hurray) but plenty of different ones. Unfortunately, the book does not cover all of the knowledge you must have to complete the problem set. I believe that several of the problems are based off material only found on the course's power-point slides.

Additionally, completing the assignment requires the acquisition of the Latex program -- to produce a pdf of all the formulas. From what I hear Latex will be easy to learn but I am still trying to grasp it. I do not believe the claims that the learning curve is only 20 minutes... Especially for math topics!

Run, Donnie.

CSCI E-168 Building Web-Based Software with Ruby and Ruby on Rails [Lecture 1]

2009-09-04T14:03:00.000-07:00

Lecture Observations

We'll learn Ruby fundamentals, meta-programming, Rails fundamentals, Rails plugins, and Rails deployment.
We will not learn File I/O, Threading, Rails Internals.
Minaswan stands for "Matz is nice so we are nice."
There are THREE Teaching Assistants!
A bunch of code was presented fairly quickly and consumed the majority of the class. It's wasn't worth taking notes about because it was all just part of the language and something you will have to learn.
It was interesting to hear that floating points must be evaluated with some rounding error, "==" will not always work correctly.
A few things that I want to learn more about were "mechanize" and the spaceship operator <=>

Observations about the class

It's a very large class where more than 40 people showed up.
The facilities were pretty nice, but not as nice as the Maxwell Dworkin building!
The professor is a very calm and conversational lecturer who prefers to go by "John."
The lecture itself was very interesting and entertaining. John puts in an effort to crack a few jokes.
Laptops were not recommended to bring to the class.

A few thoughts...

I really liked the class and did not chicken out this time to introduce myself (see my last lecture...).
It's nice to actually talk to a professor and the TA for once in my life. It isn't THAT scary.
Ruby really does have that "friendly" aura about it. It's actually kind of sad because the Java community always makes me feel un-welcomed (aka an idiot). I haven't worked with many Ruby guys (any actually) so let's hope that it is different.

Speaking of chicken

Arrested Development Chicken Dances - Watch more Funny Videos

CSCI E-168 Building Web-Based Software with Ruby and Ruby on Rails [Intro and CH 1, 2, 6, 7, 4]

2009-09-04T13:15:00.000-07:00

Introduction

I decided to take the Ruby and Rails course because it's been one of those things that I haven't gotten around to. I feel like I'm so used to Python that picking up Ruby would make me uber confused as I'm sure I'll still work on my python-google projects and want to work on a few Rails things as well.

Will this help me find a job? Who knows (I think several students come in thinking it will). What I do hope is that it will help me get into Harvard Extension's Graduate program though.

My prior experience with Ruby is from 2005 or so. I was working on a military project and wanted to try building the multi-million dollar program in a developer-friendly framework (let me tell you it's not Struts). I discovered Ruby from the beautiful Poignant Guide to Ruby and eventually found Rails from several hyped Slashdot articles.

At the time, I built a few sys-admin tools using Ruby and probably bought the very first edition of the Pragmatic Series (pre-released in PDF form). My manager didn't quite trust a framework that hasn't reached the "1.0" milestone. Everything was going really well until I discovered Python and the Django framework. I kind of liked how Django wasn't spammed on Slashdot every day and I wanted to write the Poignant Guide to Python. Ruby already had it's celebrity (_Why)!

Since then, I haven't looked back and now that I do it's a whole new world.

Chapter 1: Getting Started

The first chapter covers installing Ruby and the command prompt.
There's a bit on the Interactive Ruby shell (irb).
Ruby programs use the .rb file extension.

Chapter 2: Ruby.new

A Class followed by the new method accesses it's constructor.
Ruby supports interpolation with #{ expression } to place code within strings.
Methods start with the def keyword and is closed with the end keyword.
Most variables start with a lower case letter (a convention).
Global variables start with a $, example $global_var_name.
Instance variables start with @, example @instance_var_name.
Class variables start with @@, example @@count_of_fingers.
A ruby hash supports any object as a key.
Ruby provides shortcuts to build arrays and hashes with the [] and {} symbols.
Ha, there's millions of shortcuts including %w{} to build an array of strings.
You can create a hash with a default value with Hash.new(def_val).

Control Structures

Blocks all close with the end keyword, but there's a shortcut to use {} to create a single line block.
while line = gets {} is an interesting block because first "line = gets" occurs before the while condition is evaluated. The conditional is actually while (line) after the input is retrieved.
It's possible to place the if or while condition after the statement for brevity.
Yields are mentioned and I am confused...

Chapter 6: Standard Types

It is possible to handle rational and complex numbers. (I hope I never have to!)
You can cast a string as an integer with: Integer(some_string)
Ruby always tries to return the most natural result for an expression.
Working with numbers is just plain awesome:

25.downto(20).with_index { |num, index| puts "#{index}: #{num}" }

Interpolated code can consist of multiple lines!
There are shortcuts for creating strings, %q or %Q or %{}
Here documents allow you to wrote large blocks of text in your ruby files. Basically you define a label to start and end a chunk of text.
The Ruby String class has over 100 standard methods.
There are two types of ranges, inclusive (..) and exclusive (...). Both include the first number.
You can use ranges in conditions, to test if a number falls in a range.

Chapter 7: Regular Expressions

Not a fun chapter to read, but I'm sure it will be resourceful. The stuff covered here is pretty darn complicated.
Luckily there are plenty of Ruby conventions to allow you to perform common RE tasks.

Chapter 4: Containers, Blocks, and Iterators

Arrays hold references, not copies of objects.
You are allowed to use negative indexes and ranges to splice data.
Arrays themselves have tons of useful methods.
To loop through an array you can use the following syntax, though I think there's also a python way using a for loop.

[1,2,3].each do |num| puts num end

CSCI E-131b Communication Protocols and Internet Architectures [Lecture 1]

2009-09-03T08:26:00.000-07:00

Lecture Observations

To understand the internet architecture, you must understand the foundation.
The course is not a course on specific products (Cisco, etc).
It's also not a programming course.
We'll need to understand the error characteristics of wired vs wireless, and why to choose one over the other.
Network diagrams are always out of date, things change and boxes move.
How do telephone networks work? I don't quite know but a few students did know know the transmission rates.
Transmission rate is based on distance.
Bit errors are rare, 1 error for every million bits. As distance increases so does the probably of a bit error occurring.
It's typically that a particular server/room to only have one error in an entire 7 day span.
If an error occurs in a packet the whole packet is thrown away.
It was less expensive to layout a HUGE wire that could support 24 channels rather than to use a single wire with the latest packet switching technology.
Multiplexing is like asking the whole class to exit the front door. We would have to form a single-file to squeeze out of the room.
Circuit-switching involved physically connecting wires to form a connection from Boston > Providence > Hartford > New York
Fixed Time Division Multiplexing is easy to develop but inefficient. Especially when certain connections are not being used. It's more or less specifying a permanent time-slot for each connection.
The following are important components of a network: error correction, error detection, user interface, applications (voip, email, web), fragmentation, encryption, flow control, the physical interface, sequencing, addressing
A packet is encapsulated. It has a header and a payload, where a payload may also have it's own header and payload.

Observations about Len Evenchik and the classroom

The classrooms in the Maxwell Dworkin are very high-tech and CLEAN.
This class is going to be online-only next semester and I am so glad I am able to experience this in person.
Professor Evenchik is very clear and I think he likes getting the class involved.
The TAs were introduce and I hope to meet them during the class meetings.
Occasionally he will sneak in a joke and some humor into the lecture.
There was one 3-5 minute break during the two-hour class.
The students in the class range from young to mature and was very diverse.

A few thoughts...
I was a little intimidated during class to ask a question. I need to get over my fear of sounding "stupid". At the end of the class I wanted to introduce myself to Len and I also chickened out. I felt like just saying "Hi" would be a waste of his time.

Next class, I will try to prepare a question from the reading... Actually, I am having trouble understanding some of the exercises at the end of the chapter. I wish there were more examples.

CSCI E-131b Communication Protocols and Internet Architectures [Math Stuff]

2009-09-03T06:20:00.000-07:00

Conversions and Numbers
1 Byte = 8 bits

Mbps is Mega-bits-per-second
MB is Mega-bytes

Kilo = 10^3
Mega (Mbps) = 10^6
Giga = 10^9

Speed of Light is 2.3 * 10^8 on a wire

100ms is a good cross-country round-trip-time (RTT)
1ms is the RTT for a local area connection

Equations
Latency = Propagation * Transmit * Queue
Propagation = Distance / Speed of Light
Transmit = Size / Bandwidth

Throughput = Transfer Size / Transfer Time
Transfer Time = RTT + Transfer Size / Bandwidth

Exercises

Ch. 1
5-a.) Calculate the total time required to transfer a 1000 KB file, assuming RTT is 100 ms, packet size is 1 KB, and an initial 2 * RTT handshake is required.

First we need to know the equation for Transfer Time. It's RTT + Transfer Size / Bandwidth. We can determine the RTT parts right now.

The handshake costs = .2 seconds (2 * 100 * 1/1000)
The one way latency = .05 seconds ( RTT / 2 or (100 * 1/1000) / 2)
Note, this times how long a packet travels from start to finish.

The amount of time it takes to transfer the whole file is determined by:
Transfer Size / Bandwidth or 1,000 KB / 1.5 Mbps = 5.3 seconds. This can be calculated by:

1,000 KB / 1.5 Mbps =
8 * 1,000 * 10^3 / 1.5 * 10^6 =
8 * 10^6 / 1.5 * 10^6 =
8 / 1.5 =
I did this freehand so I determined 5.3 by dividing 16 / 3 = 5.3

Thus, adding the latency, handshake time, and how long it takes to transfer a file:
.2 s + .05 s + 5.3 s = 5.55s

It takes roughly 5.55 seconds (using a lot of approximations) to transfer the entire file.

** Will be updating throughout the course **

CSCI E-131b Communication Protocols and Internet Architectures [Intro and CH. 1]

2009-09-01T17:29:00.000-07:00

For Fall '09, Harvard Extension is offering a Communication Protocols & Internet Architectures with Len Evenchik. The meetings are Mondays from 5:30-7:30 PM.

I decided to take the class because the Web is my world and it's been a while since I understood how it actually works. Right now I see it as a bunch of TLAs (three-letter-acronyms) that scare the heck out of me. Why should I care about TCPs, UDPs and IPs? I'm just a developer!

But after working for a major newspaper and trying to run my own start-ups I realized that's an awful mindset to have. I need to have a better picture of this whole world -- wide web -- that I love to work in. At times I'm not just a back-end developer, I'm the front-end guru, or the database genius, the sys-admin g*d, etc, etc.

Now let's get started. In blogs titled "Communication Protocols and Internet Architectures" I'll be referencing the book Computer Networks by Larry L. Peterson and Bruce S. Davie (Fourth Edition).

Note: All of my bullets can probably start off with the phrase "I think that" or "I don't really know but it seems like".

Chapter 1: Foundation

A network was a term originally used to describe the relationship between remote terminals and a main-frame computer. Think using a keyboard and a LCD screen but having the Computer itself (CD-ROM, etc) 200 ft away.
Many networks are optimized for a specific task. There are telephone networks, cable networks, and satellite networks to name a few. Not all networks support computer, phone, or cable traffic as efficiently as another.

1.1 Applications

The World Wide Web (Web for short) is not the internet. The Web is an application that uses the internet.
Everything selectable item on a web page has a special identifier. It's referred to as a URL or URI (Universal Resource Locator/Identifier).
Every time you click on a URL, many messages are sent between you and remote computers. Some include establishing connections and discovering IP-addresses in order to retrieve data.

1.2 Requirements

Building the internet involves trying to please almost everyone.
Application programmers (speed), network designers (efficiency), and the network providers (administration) all have different goals.

1.2.1 Connectivity

Scale is a popular buzzword. It means to support growth and typically refers to a system built to grow-and-grow-and-grow-and-grow. The internet scales as new users and systems are added to it.
A link is a connection between computers (a cable, fiber, or wireless). Several machines sharing a link forms a switched networked.
There are two main types of switched networks, circuit-switched and packet-switched. If you've ever watched Mad Men you'll occasionally see circuit-switched networks. It's when -- way-back-when, a telephone operator would physically move widgets and doodads in order to get your call through. I wasn't alive then but I imagine it looking like:

Packet-switched refers to a trickier concept. It's where packets (blocks of data) are passed from machine to machine, at times using shared resources and links. Since ALL data cannot flow instantaneously, the data or packets "take turns."
A packet-switched network uses a store-and-forward strategy. A machine will first save the data and then pass it to the next machine. In your office you may have a piece of hardware called a switch whose primary function is to do this well.
The process of determining where certain packets go is called routing.

1.2.2. Cost-Effective Resource Sharing

Sharing a common resource such as a link is called multiplexing. It's how data from several machines can somehow "fit" on the same wire. De-multiplexing is how the same data makes it off of the wire and to its destination.
There are several ways to share a resource. The time-share way is called synchronous time-division multiplexing (STDM). As you can imagine, it is sharing based on predetermined time-slots. This can be inefficient when the time-slots are not being used.
Frequency division multiplexing (FDM) transmits data at different frequencies over the same resource. I'm believe this is how we can cable-tv, internet, and phone through the same coaxial cable.
When computers talk to each other, the data is sent in smaller blocks of data called packets. Occasionally, packets are lost or dropped when too much data is passing through a switch. Remember store-and-forward? When someone can't be stored (like a packet) it's good-bye!

1.2.3 Reliability

There are three classes of network issues: bit errors (0s turning into 1s and vice versa), lost packets, and physical issues (blue-screen-of-death).

1.3 Network Architecture

A network architecture (blueprint) helps network designers plan and implement a network.

1.3.1 Layering and Protocols

Layering allows you to modify or add functionality at a specific layer.
A protocol provides a communication service that things (higher-language-objs) can use.
A protocol has two parts/interfaces (a service and a peer interface)
The service interface defines how software on the same computer can use the protocol.
The peer interface defines how to different machines can use the protocol to communicate.
A protocol specification defines how a protocol is designed and implemented.
Interoperate is a term used to describe two protocol modules that implement a specification, thus allowing communication.

1.3.1 Encapsulation

As there are different layers, or network modules, there are specific ways to handle the piece of data that is being transferred. Encapsulation is a process of only showing a particular part of useful data to each particular module.
There are two main parts, the header and the payload.
The header is like a mailing label, it describes just enough information to know where a packet is heading and where it came from.
The payload is the data itself.
Encapsulation is a recursive process where each layer can wrap and unwrap a packet.
The demultiplexing key (demux key) seems to be an identifier to how messages were multiplexed and demultiplexed. Maybe this helps a particular layer dissemble and reassemble a set of packets?
ISO is a common term: International Standards Organization.
They helped define a common way to connect computers using an architecture called Open Systems Interconnection (OSI).
It's more or less a 7-layer system (application [...] network)

1.3.3 Internet Architecture

The internet architecture is often called the TCP/IP architecture after its two main protocols.
The protocols, when illustrated top to bottom, for an hour-glass shaped figure where everything is funneled through a common IP layer.
The protocol modules at the top (application) and bottom (network) may be interchanged, while IP is necessary for any internet traffic.
Interesting enough, the layering is not strict and modules may leap-frog over a module... An application may use the IP protocol directly.

1.4.1 Application Programming Interface (Sockets)

The Operating System (OS) provides a way to hook into and use network protocols. It's called an Application Programming Interface (API) or socket.
A protocol is a provider -- of a service -- and the API/socket is the syntax that the OS provides.

1.4.3 Protocol Implementation Issues

There are 2 main ways that an Operating System allocations resources for a network subsystem.
Process-per-protocol involves starting and stopping a process/thread for each layer or protocol module that's involved.
A protocol module is subsystem that handles a particular protocol (TCP, IP, UDP, etc)
Process-per-message involves allocating a process/thread for the message as it traverses through the entire network stack.
[??] I am confused about what this all means.
The OS maintains an abstract data type for the message as it is handled by the system. This allows for the OS to by-pass the store-and-forward process, not having to copy the message up and down the network stack.

1.5 Performance

Network designers build for performance.
Performance is measured by bandwidth and latency.
Bandwidth (heavily related to throughput) is how much data can be transmitted at a given time.
Bandwidth is constantly improving as technology improves. More bits (1s and 0s) can fit on the same wire.
Latency (delay) is how long it takes data to reach its destination.
Its more important to measure latency by it's round-trip-time RTT. This is how long data takes to reach its destination and back.
Bandwidth is more significant latency with larger file sizes. Latency is significant when fewer packets are being transferred.
The specific amount of time that it takes to send a signal from one end to the other is called the propagation-delay.

1.5.2 Delay x Bandwidth Product

If the delay is how long, and bandwidth is how much at a time, then the product is the volume (of the channel/pipe).
[??] I'm not sure if I understand the significance.

1.5.4 Application Performance Needs

Some apps, such as video, can state a ceiling on how much bandwidth is needed.
Video can be compressed and only the bits that change from frame to frame can be sent.
Jitter describes the latency between packets or the interpacket-gap.
If you know the upper and lower bounds of latency on a network, you can delay the start of a video to wait in order to stream the video without any hiccups.

1.6 Summary

Networks must be cost effective.
The layered internet architecture defines a blueprint and the protocols are the means of communication.
The socket interface is the interface between applications and the OS networking subsystem.
Networks have an emphasis on high-performance. Delay * Bandwidth is important in protocol design.

from harvard import education

2009-09-01T16:16:00.000-07:00

I recently decided to start taking some Computer-Science classes at Harvard University's Extension College. During the next three months I plan on sharing my education (as legally possible). Hint, the economy and the job market probably had something to do with it.

If you are one of my previous students, I hope you find this helpful. If you're a friend... wish me luck! I'll need it.

Goals
I am terribly excited and nervous about school. Anxiety is probably my middle name -- or is it Modest. I hope blogging about what I'm learning will help me overcome my insecurities. But that's just a secondary goal.

My main objective is to discuss the required reading and lectures -- probably through quick bulleted summaries.

Additionally, I'll do my best to simplify the material and draw pictures when I have the time.

Keeping your AppEngine projects loaded in memory

2009-08-12T08:44:00.000-07:00

After creating a new google app-engine project you might notice a delay when accessing your site. What gives?! You thought this was Google right? Stuff deployed on the app-engine should be fast...?! Right...? Am I Right?!!

Well, yes and no. To keep this simple, Google does not keep every application loaded in memory. Whenever you site has not been accessed for 5 seconds, it will be dropped. Thus, when you first go to your website it will need to loaded back in order to handle your request. This causes a unfortunate and noticeable delay. It makes showing your site to friends and colleagues a pain if it's not being constantly hammered by thousands of users. People will just assume that you code inefficiently!

"Your website is SLOW!"
"I think your queries must suck or something?"
"Maybe you have too much JavaScript?"
"Have you thought of a CDN?"

To avoid silly questions like this, you can do something. 1) Market and make your website insanely popular or 2) HAMMER IT!

Here's a super simple python script that will ping your website forever, thus keeping it loaded by Google. You should be able to run this on a computer 24/7:

import time, httplib

urls = (
"www.aboutaplace.com",
"www.canihavea.com",
"www.gaegigs.com",
"www.yurlp.com"
)

while(True):
for u in urls:
  try:
    c = httplib.HTTPConnection(u)
    c.request("HEAD", "/")
    res = c.getresponse()
    print "touched %s, %s %s" % (u, res.status, res.reason)
  except:
    print "could not touch %s" % u
time.sleep(3)

To make this work for you, change the urls to your own domain. I suggest making HEAD requests to keep the bandwidth minimal.

Wave Rocks and Sucks because...

2009-08-04T15:56:00.001-07:00

So I was playing with wave and I was actually able to delete a few people's conversations... Sorry! I really didn't mean to... *sigh*

While perusing a waves I came across one named Discoveries and Observations about Wave and it's worth sharing with the rest of the world. Personally, I can see why some people love wave but I don't see it being all that and a bag of chips the way it is right now.

WAVE ROCKS BECAUSE...

The pattern of write-reply communication is rampant in today's internet software solutions. Blogs, forums, chat, social sites, the classic "guest book", etc. are all write-reply patterns. Wave builds on this pattern while leveraging today's browser technologies and the user's higher bandwidth connection to both consolidate these technologies into an open platform for real-time, social communication.

Mixing realtime IM/Threading/document-collaboration is a great idea.

Aug 2
Mario Viviani:
in my opinion Wave is going to be the Super-Social Network. I mean that this stream-oriented flux of informations will obviously be used to share "social" informations. Think about organizing a meeting with your friends using Wave: it's much less complicated than using Facebook or Twitter.. With wave it's easier discuss about a video or a link.. In this way, i think Wave is going to be super-used by actual Social Network users.

Aug 2
Mickael Faivre-Maçon:
right now, facebook is easier than wave.... I don't see wave as a social network thingy.

Wave as an open standard will revolutionise the interwebs.
Wave can become a platform to collaborate and communicate in for other applications.
Can make other applications easily accessible over the web in standardized way.

Aug 1
Omar Balbuena:
I see. I don't understand what is being standardized though. Maybe user credentials?
I just find it weird to see "new features" and "standardization" in the same sentence, it's an oxymoron and it still doesn't make sense to me. Familiar interface maybe?

Having multiple people working on one document will decrease work time ten-fold!
Seeing exactly what people are working on will save time.
Will remove necessity of having multiple communication platforms (im, e-mail, forums, letters, smoke signals)
You can read while they are typing, conversations become much quicker.
It puts almost no constraints in how you shape communication at any point in time (an email can become a conversation can become a chat can become a document can become a game)

WAVE SUCKS BECAUSE...

Robots can fundamentally break the wave experience (e.g. groupy)
And there is currently no way of removing ones that you add.
Permissions and wave management (kicking people etc...) are currently too limited
It's bewildering; I can't see the things that are interesting to me or get rid (easily or obviously) of the things that I don't care about
Difficult to give priority to different waves. Some may need attention more than others.
Wave is more complicated than alternatives such as email/forums etc... It risks alienating the less technically capable.
Needs more visual cues for when/where other editors are when editing the wave document.
Needs more server CPU added in current dev preview :)
Puts a lot of stress on the Javascript engine (currently works better using v8, no surprise there)
Though when Wave becomes an open standard, we can hope to see non-browser implementations also
No chrome on Mac yet, so can't test... http://www.google.com/chrome/intl/en/eula_dev.html?dl=mac go download it :D
I tested Wave in MacOS Chrome and it crashed after 2 min. Too early for the transation weight of Wave
Refactoring threaded conversations is painful with the existing wave client.
Drag/drop restructuring of threads and easier deletion of wavelets would be handy (via a keyboard shortcut).
It may be difficult to get wide adoption because people will have to learn something new, and that in turn may significantly decrease usefulness of the platform.
Feels wrong, in such an application-like environment, to not have right click menus
Too hard to see who has edited what.
it hammers the hell out of my firefox
it's difficult to see when it's doing something, like is it fetching data or just sleeping. Feels like it needs a progress bar
It puts almost no constraints in how you shape communication at any point in time (you never know what you are supposed to be doing)

Jul 31
Omar Balbuena:
I don't see why that is bad.

Jul 31
Anton Babushkin:
Having no clear direction is always bad, especially for those who are simply "meh" and don't want to put too much thought into things (i.e. your regular user)

Is essentially a new type of mail, yet, isn't backwards compatible (e.g. being able to send emails with it would be useful for those without wave)
Much information is lost when the wave turns into a rant, could be a problem for longer waves.
Can't get to the newest item in a Wave easily. When there's lots of items, it would be convinient for a keyboard shortcut (left arrow?) to go to the newest item, which may, in some cases, not be the bottom.
I don't like scroll bars but I don't like that my cursor keys don't work line at a time. They're more like page keys. Fine on the small entries but entries like this one that are bigger than a page, it fails badly.
I hate to scroll back to find edit/reply button when the comment is too long. ditto 'done' button at bottom.
The search isn't good enough. If I was adding someone called "Fred" to a Wave, and my criteria was "fred" then it wouldn't find "Fred"
Hard to see what's new in a wave when there has been a lot of edits
Navigation (e.g. the center list of waves, content on the right) is a little archaic feeling.

Thank you Dion Hinchcliffe for starting this wave. I agree with most of your points.

Google Wave Frequently Asked Questions (FAQ)

2009-08-04T11:00:00.000-07:00

I found this on a Wave and though it was worth putting up on a public blog. Cheers!

WaveFAQ

This is a public FAQ. Please feel free to edit the top portion, keeping it clean and concise, deleting redundant text. As Wikipedia says, "Be Bold".

The content of this FAQ is licensed under the Public Domain, so as to be compatible with Wikipedia et al. You are considered to agree to that by editing here.

If you really want to add a wave comment or reply, do so at the bottom, rather than inserting it inline in the actual FAQ where it will break the flow. But in Wikipedia style, we'll probably want to split commentary off to another "discussion" wave somehow, so don't count on your comments staying here.

Getting started

Why don't I know what to do when I get here?

It was difficult to figure out this much. It appears we need a welcome wave when we join the service to understand how it works.

Welcome to Wave has links to several other getting started waves created by Wave developers to help you learn the UI, keyboard shortcuts, how to use search to manage your workspace.

As of July 23, search for public waves has been implemented. For more info try these searches:

with:public tag:welcomewaves

with:public tag:help

with:public tag:faq

Where is the documentation?

Resources within the Sandbox.

with:public tag:welcomewaves

Resources Outside of the Sandbox

Google Wave API Overview

Wave Extensions

Google Wave Federation Protocol

Where is the sample code?

http://code.google.com/apis/wave/samples/index.html

Sharing waves

Public waves

A public wave is a wave that all users with a sandbox account can access even if they are not explicitly added as participants of the wave.

Note: If you want to see updates to a public wave in your inbox, you must become a participant of the wave. Another option (see below) is to define and save a search that would return the wave and check Searches folders instead of the inbox.

How do you make a wave public?

To allow all users access to your wave, add the special address public@a.gwave.com as a participant of the wave.

Note: You can link to a public wave and anyone [with a sandbox account?] will be able to open it and participate in it. However, unlike sending it to them directly or via a group it does not spam their Inbox.

How do we find/lookup this user?

You need to add public@a.gwave.com to your contacts (either through the external contacts manager or through the wave client).

How do you then share that wave with other users? Ctrl+L and wave ID to create a link?

[If you are using Chrome] you can drag and drop your wave into other waves to create a hyperlink.

It's also possible to select "get wave ID" from the Debug menu at right top of the page. Copy this url, mark some text you want to transform into the wave link and press ctrl+l. You will get a popup window where you can paste your wave ID. Note: You must be in edit mode to do this.

How can I find other public waves?

As of July 23, public waves are searchable.

with:public

You can refine the search in various ways. For example,

with:public tag:help

with:public about:avatar

What does yellow box 'The wave is published at an unknown url.' at the top of the wave mean?

It means that the wave has a public url.

Why do I get the message "You are not a participant in this wave"?

The wave is not a public wave and you are not a participant.

The link to the wave is wrong.

You are a participant via the wave-discuss group but you were not a member of the wave-discuss group at the time the wave-discuss group was added to the wave.

Embedding waves

How do I embed a Wave to my blog?

Check out Wave Embed API Tutorial

Is it true that embedded wave can only be viewed in browsers with Google Gears >=0.5.21.0?

No, you can also view it with earlier versions (0.5.19.0 works for example), but drag-and-dropping images is not supported in earlier versions.

How do you add yourself to a wave (like this one) without editing it or adding a comment?

Click the "+Add" button that is in the top bar with all of the other participants' avatars. Type your own username in the search box that appears and hit enter.

Is an embedded Wave viewable by non-Wave users?

No. Currently, you need a Wave account to view the embedded Waves.

Future?

What will happen to our sandbox accounts when Wave goes public?

Brian Kennish: We're going to try to migrate data, but no guarantees. You should treat everything in the sandbox as though it might get deleted.

Can my other friends who did not go to Google I/O sign up to use the dev preview of Wave?

Have them fill out the Google Wave Sandbox Account Request and the Wave devs will get them on as soon as possible (which will likely be a few weeks at the earliest). Source: Gregory Dalesandrev

Browsers

What browser release should I be on?

Chrome

You can switch between the chrome "channels" (stable, beta, dev) using the channel changer.

Firefox

3.0+ is recommended, though 2.0 seems to work (though not that well)

Safari

4.0 seems to work okay

iPhone 3G2 V. 3.2 works okay, little slow

Internet Explorer and Opera are not supported

Midori (linux experimental build )

Editing in a wave

Why is there no "modeless" editing?

I have to click the pen at the top of the wavelet and then the done button on the bottom. Isn't that unnecessary?

Why is still unknown. However keyboard shortcuts ease the pain. For example:

To edit a document (blip), select it and press Ctrl-e.

To save your edits, press Shift-enter.

Are there any keyboard shortcuts?

Yes. See the Keyboard Shortcuts Cheat Sheet.

Edit Toolbar

Add your comments to the Issue 78 at google-wave-resources to show support for a floating edit toolbar.

Links

How do I add a link?

Click on the G+ tool and search for it.

But how do I insert a link when I already know the URL?

Select some text, then hit Ctrl+L and it will open a window where you can paste a save id or URL.

If you want to link to another wave, open a document (blip) in Edit mode and drag a wave into it. [Chrome only?]

If I can't drag a wave to create a link, how do I find the wave ID to link to?

Get the wave ID from the Debug menu.

How do I remove a link?

You edit the wave and delete it.

Google Wave First Impressions, Sorry Google!

2009-08-04T09:59:00.000-07:00

I recently logged onto Google Wave with my developer's key and here are my initial and raw impressions. Please keep in mind that Wave is in a development state and any of the issues I may be mentioning will be resolved eventually!

LOGGING IN
The log-in process is relatively easy. Google e-mailed me my user name and password and there were no issues. I would like to note that you if you try to use your full Wave e-mail address you will be prompted with a log-in error.

THE WAVE DASHBOARD AND LAYOUT

When you log into Wave, you're redirected to the dashboard. It's very similar to the G-mail process but a LOT more sluggish. When Wave first loaded on my Ubuntu laptop, I was prompted with an error where Google wanted to know what I did to break the 'experience.' I believe I wrote:

I just logged in. For some reason it broke.

Okay, so logging back in the first thing I noticed was how 'sluggish' the interface felt. I opened Wave on my MacBook and noticed the same feeling. I am sure this will be improved. However, one glaring issue that I saw was how Wave looks a lot like OutLook/Entourage. Within your middle-well you have all of your messages and on the right-well you have the message that you've opened. This might work well on desktops but not on a laptop. With my 13", 15", and 17" screens and playing around clicking on messages this was an AWFUL experience. G-mail works because each message utilizes the majority of your screen to read a message. I hope Wave becomes more laptop/smaller screen friendly in the future.

MAILBOX OF SPAM

Again, I'm new to Wave and I may not understand how everything currently works. It is wierd that when you log in you have thousands of 'waves' in your mailbox. I guess it's similar to connecting to a newsgroup but I definitely was not expecting that. I suppose I assumed it would look and feel like 'gmail'.

This is annoying to me because a significant portion of these 'waves' in my mailbox are unreadable! If you try to click on them, you will be prompted with:

Why show someone an e-mail if they can't read it? If I'm not a participant for a wave, then how can I request access? It wasn't obvious where to click or go. That said, it also isn't clear how to delete a wave (e-mail) that you created.

SLUGGISHNESS

Unfortunately, maybe playing with Wave at 11:30 AM on a Tuesday was a bad thing to do. Not only did the interface act unresponsive, the glitches were popping up all over the place. Yay, development stage! :-)

SUGGESSTIONS

I think Wave may become an invaluable tool but I don't see it yet. My biggest gripe is the User Interface and I wish it was a lot less cluttered. The G-mail UI is excellent probably because it's simple and makes sense. Wave's UI does not make sense and I doubt that my Mother or technical Father would be able to log in and know what to do. My initial suggestions are:

I would like the option for messages to open up and fill the full screen.
Checkboxes would make sense for tasks like deleting or moving messages, why aren't they there?
Only show public messages (things you can read) in your mailbox.
Provide an "e-mail" mode, allowing G-mailers a way to adapt to Wave.

Google App Engine (Java Flavor) and Google Wave

2009-07-29T17:09:00.000-07:00

Yesterday I began my journey using the Java-version of the Google App Engine. I love the Python implementation but I have noticed there are much more Java jobs out there than Python jobs. It's super sad but true. Actually I guess it would be hard to say there are "jobs" out there.

I think my first project will be gaegigs.com. Yes, I know the domain name sounds funny but it will be a simple job board for Google App Engine developers and projects. Alternatives would be rubynow.com and djangogigs.com. Maybe this GAE thing will take off some day?

I recently completed the tutorial for the GAE-Java flavor and hoisted it up onto gaegigs.com. I'm still learning the ins-and-outs so right now the site may be a simple guest book.

Additionally, I recently received my Google Wave API key! I can't wait to start playing around with it -- time, I need more of it. Any requests? I'm curious about building a commenting system with it that could trail blog or story postings.

Creating or Editing a Java File

2009-07-29T08:27:00.001-07:00

This is part in a series where I blog some of the common threads from my Intro to Programming course.

Writing a Java program requires the use of a text-editor. And any will do actually.

If you're using Linux based system, I'm sure you're already familar with using an editor. However, for my Windows-based students this may be new territory.

A text-editor that you may be familiar with is MS Word. Word, like many document editors offer various helpful features. When you save a document in Word, it generates a file with an extension of ".doc" that you can move or re-open later. HOWEVER, we're writing programs and we need an editor that actually helps us write a file with a ".java" file extension. Please do not use MS Word to write your programs :-)

Simple text-editors that you may use are: Notepad, Notepad++, and ConTEXT among many others.

Notepad: Notepad is already installed with Windows. You can access it from the Start menu or use the following commands: "cntrl-r: then type notepad". If you try to write a Java program with notepad, make sure that the file is saved as a ".java" program. Notepad will always try to save a new file as ".txt", so the first save may look like "Assignment1.java.txt". You will need to rename that file through the explorer or command prompt.

Notepad++: From the website - "Notepad++ is a free (as in "free speech" and also as in "free beer") source code editor and Notepad replacement that supports several languages. Running in the MS Windows environment, its use is governed by GPL Licence.

Based on a powerful editing component Scintilla, Notepad++ is written in C++ and uses pure Win32 API and STL which ensures a higher execution speed and smaller program size. By optimizing as many routines as possible without losing user friendlyness, Notepad++ is trying to reduce the world carbon dioxide emissions. When using less CPU power, the PC can throttle down and reduce power consumption, resulting in a greener environment."

http://notepad-plus.sourceforge.net/uk/site.htm

ConTEXT: From the website - "is a small, fast and powerful freeware text editor, developed to serve as a secondary tool for software developers. After years searching for a suitable Windows text editor, we didn't find any of them to satisfy our needs, so we wrote our own."

http://www.contexteditor.org/

What editor do you use or recommend? I personally use Text-Mate or Vi. Occasionally I'll use Eclipse (it's way too powerful for a new programmer though).

Jeff mentioned:

My computer is a Mac, and I'm using a text editor called SubEthaEdit. I recommend using it because is has syntax highlighting, which puts the Java keywords and comments in different colors. I'm using SubEthaEdit version 1.1.5 (version 2.2 didn't work on my computer). I downloaded it for free at http://www.codingmonkeys.de/subethaedit/old.html

Campbell and Stephen wrote:

I found a decent integrated development environment program that is a free download that works well with Java. You can edit java files and compile them from the same window, and it automatically formats your indents from line to line.
http://sourceforge.net/projects/scintilla/files/SciTE/Sc177.exe/download

So you've written a .java file, now what?

2009-07-29T08:23:00.000-07:00

This is part of a series where I am posting a few of my common threads from the Introduction to Programming (Java) course that I'm teaching.

After writing a .java file, there are a few steps that must take place in order for you to run your code. Let's use the file "MyProgram.java" for the following example.

Step 1: Locate your .java file. Where did you save the file? Let's pretend that the file exists here "c:/JavaPrograms/MyProgram.java"

Step 2: Open up your command prompt. A shortcut is to press "control-R" and type "cmd". (Windows users only)

Step 3: Change Directory to the .java file. In the command prompt you can type "cd c:/JavaPrograms/"

Step 4: Compile the Java file. In the command prompt you can type "javac MyProgram.java"

Step 5: Fix and Errors. If step 4 produces and errors, you must fix them. To fix an error, you must open up the .java file in an editor and make sure that the code is correct. Typical errors are missing symbols and typos. After you ATTEMPT to fix the errors, you must re-compile again until all of the errors are gone.

Step 6: Run the program. If there were no compile errors, a .class file will be generated. It may look like "c:/JavaPrograms/MyProgram.class" and exist in the same directory. To run the file, you can type the following on your command prompt "java MyProgram"

The Command Prompt

2009-07-29T08:15:00.000-07:00

There are a few threads I use every semester and I will be posting them on my blog for future reference. This blog covers accessing the command prompt.

To open the command prompt in windows:

Click on the Start menu button
Click on the Run.. button
Type in "cmd" and push OK
OR push the Windows-Key and the R button simultaneously
and type in "cmd" and push OK

To open the Terminal in OSX:

Open the Finder
Click on Applications
Click on Utilities
Then double-click on Terminal
It's useful to add Terminal to a more convenient location, to do so just drag and drop it onto your dock

Some of you may think that using the shell (command prompt/terminal) is the OLD way of using computers. Now-a-days, we use a window manager and graphical user interface such as Windows, Linux, or OSX. However, as a programmer its fairly typical to spend a lot of your time on a command-line interface. In this course it's fairly important because the The command-line is important in this course because it will help you compile, debug, and run your programs.

List the files in the current directory:
{WINDOWS} C:\DOCUME~1\Student > dir
{LINUX/OS} $ ls

Change to a new directory:
> cd folder1

Go back one directory:
List the files in the current directory:
> cd ..

Rename a file, from MyApp.java.txt to MyApp.java:
{WINDOWS} C:\DOCUME~1\Student > rename MyApp.java.txt MyApp.java
{LINUX/OS} $ mv MyApp.java.txt MyApp.java

Compile a Java file (.java) to ByteCode:
> javac MyApp.java

Run a Java program (after it's been compiled):
> java MyApp

In the Summer 2009 course, Andrew pointed out:

I'd also like to share the following keyboard shortcuts for the command prompt. Using the Up/Down arrow keys to recall previous entries is very helpful.
(Source: http://forums.techarena.in/tips-tweaks/1151516.htm )

F1 – Repeat text typed in preceding line, one character at a time
F2 + Key – Repeat text typed in preceding line, up to first character matching key
F3 – Repeat text typed in preceding line
F6 – Place an end of file character (ˆZ) at current position of command line
F7 – Show all entries in command buffer
Alt + F7 – Clear all entries in command buffer
Ctrl + C – Used to interrupt the output of most command prompt applications
Left Arrow – Move back one character
Right Arrow – Move forward one character
Ctrl + Left Arrow – Move back one word
Ctrl + Right Arrow – Move forward one word
Up Arrow – Display previous command entered
Down Arrow – Display next command entered
Insert – Toggle insert/overtype mode (block cursor implies overtype mode)
Page Up – Display oldest command entered
Page Down – Display newest command entered
Home – Move to beginning of line
End – Move to the end of line
Esc – Erase current line

Python Generators in Plain English

2009-07-28T22:10:00.000-07:00

A buddy of mine needed help understanding what a generator was in Python. He tried searching the web but unfortunately the web is not kind to the non-uber-technical.

If you don't want to read, here's a summary:

a generator vs a list is more memory friendly
you can create a generator using parenthesis or defining a method with the yield keyword

In order to understand what a generator is, you need to start with an iterator. In Python, an Iterator is an object that has two main functions, __iter__() and next(). The whole purpose of this object is to return an element from a list or collection of objects.

Every collection-class and list has an iterator object that's accessible through the __iter__() method. In addition, whenever you use the for loop statement you actually call __iter__() on the collection following the in keyword.

The code:

for i in [1,2,3]:
 print i

roughly translates to:

iterator = [1,2,3].__iter__()  # or iter([1,2,3])
while True:
 try: print iterator.next()
 except: break

As most people define it, a generator is a way to automatically create an iterator. That may confuse the heck out of you but let's think about it. An iterator is a way to access each individual thingy in a container right? By default it seems that Python provides us those iterators with the typical square-brackety list objects that we all love.

Now what if you wanted to go through a particular object but you had special conditions with how you wanted access the data. A good example is trying to determine the first 20 Fibonacci numbers. It's common to program the Fibonacci sequence using a recursive method, so let's compare it to using a generator.

But first, let's look at how you create a generator using the yield keyword. The yield keyword allows you to return data within a function and return to this statement when the subsequent next() method is called.

>>> def hello_world_generator():
... yield 'hello'
... yield 'world'
...
>>> iterator = hello_world_generator()
>>> iterator.next()
'hello'
>>> iterator.next()
'world'
>>> iterator.next()
Traceback (most recent call last):
File "", line 1, in
StopIteration

Notice how the first time you call next on the iterator, the "hello" string was printed. Then on the subsequent next call, "world" was printed. With that, we move onto the Fibonacci example.

>>> def fibo():
... a, b, n = 0, 1, 1
... while n <= 20:
... yield a
... a, b = b, a+b
... n += 1
...
>>> def fibo_recursive(n):
... if n < 2: return n
... else: return fibo_recursive(n-2) + fibo_recursive(n-1)
...
>>> import datetime
>>> start = datetime.datetime.now().microsecond
>>> for i in fibo():
... print i
...
0
1
1
2
3
5
8
13
21
34
55
89
144
233
377
610
987
1597
2584
4181
>>> print (datetime.datetime.now().microsecond - start)
2380
>>>
>>> start = datetime.datetime.now().microsecond
>>> for i in range(0,20):
... print fibo_recursive(i)
...
0
1
1
2
3
5
8
13
21
34
55
89
144
233
377
610
987
1597
2584
4181
>>> print (datetime.datetime.now().microsecond - start)
10519
>>>

Boy what a difference! In this simple case, using a generator was about 5 times faster. Basically, a generator allows you to dictate how you will examine a collection without storing everything upfront in memory. When working with large sets, or stacks as seen with the Fibonacci sequence, a generator could be very useful. Only evaluate where you're currently at in a collection (or method) and remember where you left off.

WGBH Redesign

2009-07-23T19:50:00.000-07:00

Tomorrow, I'll get the opportunity to talk to the WGBH folks. While doing a little bit of research I noticed that their homepage looked a little stale and decided to procrastinate and redesign it. It's not really creativity but I think it would look great with a more modern feel.

Getting Virtualenv to Run on Your Mac

2009-07-23T11:15:00.000-07:00

I've seen a lot of chatter on the django-user-groups about using the virtualenv utility. If you're running multiple apps on various environments, maybe one on Django v96 and something slightly newer on Django v1.02, virtualenv can be a lifesaver.

In the past I would download and setup various packages while not installing them globally. Thus, whenever working on a particular app I would have to make sure to change my PYTHONPATH with various set/alias commands. Now, with virtualenv, it's a little easier to develop an app with its own local environment. Each "virtual environment" you will have your own python executable and site-packages, keeping everything nice and isolated when you're testing out new doo-dads.

INSTALLING VIRTUALENV
Here, I'll go over the few steps needed to get virtualenv up and running on a Mac. First install pip. Then on the terminal type the following:

donnie-demuths-macbook-air:Sites donnie$ sudo pip install virtualenv
Password: *******
Downloading/unpacking virtualenv
Downloading virtualenv-1.3.3.tar.gz (1.0Mb): 1.0Mb downloaded
Running setup.py egg_info for package virtualenv
Installing collected packages: virtualenv
Running setup.py install for virtualenv
Installing virtualenv script to /usr/local/bin
Successfully installed virtualenv
donnie-demuths-macbook-air:Sites donnie$

Unfortunately, if you don't have xcode installed yet you will need to get it. It's a real pain because the file is so HUGE (2.5GB+) and it's mandatory. You can get evil xcode here but be sure to remember you apple ID and password. Btw, I hate any mandatory huge files that I don't really want.

CREATING A PYTHON VIRTUAL ENVIRONMENT
Using virtualenv may seem confusing and will probably take a bit of tinkering around. I suggest trying the following:

virtualenv MyFirstEnv

This will create MyFirstEnv/lib/python2.X/site-packages and MyFirstEnv/bin/python. To start using this environment, use the source command.

source MyFirstEnv/bin/activate

Now you when install any new packages using easy_install or pip it will be installed to the MyFirstEnv environment rather than globally! To get out of this environment, you can go back to global by using the deactivate command.

deactivate

MORE REFERENCES
Working with virtualenv -- Arthur Koziel
Justin Driscoll: Setting Up A Django Development Virtual Environment on OS X

Installing pip and ipython on a Mac

2009-07-23T10:11:00.000-07:00

Way back when I met Alex Clemesha and Dorian Raymer, two talented surfers/pythonistas, who recommended iPython. For some reason I never knew of it and it became an indepensible tool when hacking through the shell. Note: these guys are the masterminds behind CodeNode and WikipediaGame and I expect to read about these guys in the future.

In this posting, I want to show you how to install ipython on your Mac using pip (a utility for installing python packages).

GETTING RIGHT TO IT
First, let's get our pip on. Installing pip is really easy and it actually uses the easy_install python command itself. Open your terminal and type in:

donnie-demuths-macbook-air:Sites donnie$ sudo easy_install pip
Password: *******
Searching for pip
Best match: pip 0.4
Processing pip-0.4-py2.5.egg
pip 0.4 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin

Using /Library/Python/2.5/site-packages/pip-0.4-py2.5.egg
Processing dependencies for pip
Finished processing dependencies for pip
donnie-demuths-macbook-air:Sites donnie$

Now we can install ipython using the pip install command. Again, in your terminal type:

donnie-demuths-macbook-air:Sites donnie$ sudo pip install ipython
Downloading/unpacking ipython
Downloading ipython-0.9.1.tar.gz (2.8Mb): 2.8Mb downloaded
In the tar file /tmp/tmpNxQZPw/ipython-0.9.1.tar.gz the member ipython-0.9.1/IPython/testing/attic/ipdtlib.py
Running setup.py egg_info for package ipython
Installing collected packages: ipython
Running setup.py install for ipython
Installing iptest script to /usr/local/bin
Installing ipythonx script to /usr/local/bin
Installing ipcluster script to /usr/local/bin
Installing ipython script to /usr/local/bin
Installing pycolor script to /usr/local/bin
Installing ipcontroller script to /usr/local/bin
Installing ipengine script to /usr/local/bin
Successfully installed ipython
donnie-demuths-macbook-air:Sites donnie$

Now you can use ipython by typing ipython.

OKAY SO WHY SHOULD I DO THIS?
Q. So why should I use pip?
A. pip is the new hotness. It's a huge upgrade over easy_install and makes sure you have all of the dependencies before it tries to install a package. We (royal "we") have all tried to install a package that just would not work: missing dependencies, corrupt files, poor error messages. Nope, that's not going to happen anymore!

Q. And what's the benefit of using ipython?
A. It provides a lot of extra features not included in the basic python shell. Some are syntax higlighting, tab-completion, and being able to man (or find more info) about a particular variable. Check it out...

A Puzzling Take Home Interview Assignment

2009-07-22T19:27:00.000-07:00

If you really want a challenge, give the following a shot. I received it a while ago and I thought it was quite challenging... and oddly fun. For this assignment I was required to write a Java program to solve the puzzle. My response, TeamBuilder.java, was received as adequate.

I'm not sure where this puzzle originated, if anyone knows please let me know.

The TeamBuilder Puzzle
You are arranging a weird game for a team building exercise. In this game there are certain locations that people can stand at, and from each location there are paths that lead to other locations, but there are not necessarily paths that lead directly back. You have everything set up, but you need to know two important numbers. There might be some locations from which every other location can be reached. There might also be locations that can be reached from every other location. You need to know how many of each of these there are.

Create a class TeamBuilder with a method specialLocations that takes a String[] paths that describes the way the locations have been connected, and returns a int[] with exactly two elements, the first one is the number of locations that can reach all other locations, and the second one is the number of locations that are reachable by all other locations. Each element of paths will be a String containing as many characters as there are elements in paths. The i-th element of paths (beginning with the 0-th element) will contain a '1' (all quotes are for clarity only) in position j if there is a path that leads directly from i to j, and a '0' if there is not a path that leads directly from i to j.

Definitions
Class: TeamBuilder
Method: specialLocations
Parameters: String[]
Returns: int[]
Method signature: int[] specialLocations(String[] paths)
(be sure your method is public)

Constraints

paths will contain between 2 and 50 elements, inclusive.
Each element of paths will contain N characters, where N is the number of elements of paths.
Each element of paths will contain only the characters '0' and '1'.
The i-th element of paths will contain a zero in the i-th position.

Examples

input: {"010","000","110"}
output: { 1, 1 }

Locations 0 and 2 can both reach location 1, and location 2 can reach both of the other locations, so we return {1,1}.

input: {"0010","1000","1100","1000"}
output: { 1, 3 }

Only location 3 is able to reach all of the other locations, but it must take more than one path to reach locations 1 and 2. Locations 0, 1, and 2 are reachable by all other locations. The method returns {1,3}.

input: {"01000","00100","00010","00001","10000"}
output: { 5, 5 }

Each location can reach one other, and the last one can reach the first, so all of them can reach all of the others.

input: {"0110000","1000100","0000001","0010000","0110000","1000010","0001000"}
output: { 1, 3 }

Stuff in a Cup, A Logic Interview Question

2009-07-22T17:09:00.000-07:00

Okay, so the interviewer didn't prompt me with "stuff in a cup." I'm certain it was marbles, pebbles, or maybe it was coins. Anyhoo, the question was:

What's the fastest way to determine which cup has the heavier stuff?

There are 9 cups
8 of the cups have coins that weight exactly 1.0 grams each
1 cup has coins that weight 1.1 grams each
Each cup has an infinite number of items
Using a scale to weigh an item counts as a move

The key word in this probably "fastest," as in how can we find the answer in as little moves as possible. My first, and naive, attempt was:

First survey half of the population by collecting an item from the first 4 cups. Weigh all 4 of the elements and if the sum contains (.1) then take a item from half, or 2, of the cups from this group. On the other hand, if (.1) does not exist in the set, gather items from the other half. Eventually, using a logarithm algorithm like this you'll find the answer without weighing an item from each cup.

Obviously there's a better way. I found out that you can write an algorithm to solve this in O(1), or simply a constant, move. This slightly better approach is:

Pull 1 item from the first cup. Then pull 2 items from the second cup... ...and finally pull 9 items from the ninth cup. Combine all of the items and weigh them all together. By looking at the sum you can determine which cup contained the (1.1) element because the possible sums will range from (45.1 to 45.9). If the sum is (45.6) then you can assume that the (1.1) item came from the 6th cup.

Unfortunately, there's more! What if the cups contained items that had the following weights:

cup 1: 1.3 g
cup 2: 8.2 g
cup 3: 1.9 g
...
cup 8: 2.2 g
cup 9: x.y g

How would you find the cup that has the weight of x.y in O(1), using a single weighing? This gets a little trickier but it's possible. The previous solution will not work because the sum would be much more dynamic. The correct way to solve this is:

Pull 10 items from cup 1
Pull 1000 items from cup 2
Pull 100000 items from cup 3
...
Pull 10 ^ (15) items from cup 8
Pull 10 ^ (17) items from cup 9
or
Pull 10 ^ (n*2 - 1) items from cup n

And weigh all of the items together in one fell swoop. The sum will be a very large number, but in a single move we'll find our answer. It might look like: xy22[...]198213

Then by looking at each pair of numbers, you can determine which cup a particular pair of numbers came from. (xy) came from the 9th and (19) came from the 3rd cup.

If the weights of the items changed from tenths to hundredths or thousandths then use 100 ^ (n*2-1) or 1000 ^ (n*21) accordingly.

The Fibonacci Number, A Logic Interview Question

2009-07-22T16:12:00.000-07:00

I've seen a share of logic questions recently and I'd like to share and discuss them. Obviously, my answers aren't the best out there but you may be able to use them to help you find some direction.

In an interview I was asked to program the Fibonacci Sequence/Number. Slow as I am sometimes, it took me a while to remember what a Fibonacci number was. In general there are three rules:

1) The Fibonacci number of 0 is 0.
2) The Fibonacci number of 1 is 1.
3) The Fibonacci number of any other number is determined by the function:

F(n) = F(n-1) + F(n-2)

That is the Fibonacci number of 2 is equal to the Fibonacci number of 1 added to the Fibonacci number of 0. Or a Fibonacci number can be determined by the addition of the two Fibonacci numbers before it (other than 0 or 1). This is all well defined on Wikipedia.

So let's look at a series of Fibonacci numbers.

f(0) = 0
f(1) = 1
f(2) = 1 + 0 = 1
f(3) = 1 + 1 = 2
f(4) = 2 + 1 = 3
f(5) = 3+ 2 = 5
f(6) = 5 + 3 = 8
f(7) = 8 + 5 = 13

Now before we look at the code, let's go over the business logic or rules. It's super easy.

We'll need a function that returns an int and accepts an int as an argument
When you pass in the value of 0, return the value of 0
When you pass in the value of 1, return the value of 1
When you pass in the value of anything else, return the value of the method with n-1 added to the value of the method with n-2

A Java implementation of this using recursion is:

class FiboTest {
public static int fibo(int num) {
   if (num == 0) return 0;
   else if (num == 1) return 1;
   else {
     return fibo(num-1) + fibo(num-2);
   }
}

//  This will test out the fibonacci numbers for 0, 1, 2, 3, 4, 5, 10, and 20
public static void main(String args[]) {
   System.out.println(fibo(0));
   System.out.println(fibo(1));
   System.out.println(fibo(2));
   System.out.println(fibo(3));
   System.out.println(fibo(4));
   System.out.println(fibo(5));
   System.out.println(fibo(10));
   System.out.println(fibo(20));
}
}

In my Intro-to-Programming course, I presented this problem to my students. One of them attempted to program it without using recursion and opting for a simple iterative loop. His code looks like:

import java.util.Scanner;
public class Fibonacci
{
public static void main(String args[])
{
Scanner input = new Scanner(System.in);

// Time to initialize a bunch of variables
int countto;
int counting;
int num1L;
int num2L;
int num3L;

// Starting values
counting = 1;
num1L = 0;
num2L = 0;
System.out.print ("What number in the fibonacci sequence do you want? ");
countto = input.nextInt();

if (countto == 0)
num3L = 0;
else
num3L = 1;

while (countto > counting)
{
num1L = num2L;
num2L = num3L;
num3L = num1L+num2L;
counting = counting+1;
}
System.out.println(num3L);
}
}
}

Want more interview questions? I've blogged recently about some of the questions I've been asked. You might want to glance over them to what employers are asking these days:
Interview Questions from a Government-Influenced Company
Interview Questions from a Start-Up
Interview Questions from a Mid-Sized Company

Interview Questions from a Government-Influenced Company

2009-07-22T14:50:00.000-07:00

Occasionally the same questions come up but it's nice when you get a few new ones thrown your way. Here are several questions that were prompted for a generic Software Engineering position. This company deals primarily with the government and military.

GENERAL QUESTIONS
Q. How do you analyze, or time/measure, algorithms?
A. The interviewer stated that a person that's been working probably wouldn't understand this question. The answer she was looking for was Big-O-Notation. Basically Big-O-Notation is used to measure the growth or how long something of size "n" may take. Constant, logarithmic, and linear algorithms would be defined as O(1), O(log n), and O(n) respectively.

Q. Are you familiar with the acronym SBIR?
A. I did not but found out that the government provides financial support to businesses engaged in research and development for government applications. The term itself stands for Small Business Innovation Research. A quick google search provided the following SBIR documentation.

JAVA QUESTIONS
Q. What is inheritance?
A. Inheritance allows you to define a class with the default behavior and fields of another pre-existing class. When declaring a class you would use the extends keyword. Java only supports single-inheritance (you can only inherit from one class).

Q. What is an abstract class?
A. An abstract class can not be instantiated and is a class declared with the abstract keyword. In general this allows you to provide a general blueprint for other classes. The methods defined as abstract within an abstract class must be overwritten by any of its subclasses.

Q. Can you have an abstract method within a non-abstract (normal) class?
A. No. Any class that has abstract methods must also be defined as abstract.

Q. What is the difference between an Abstract class and an Interface?
A. A typical abstract class may or may not have abstract methods. An interface only has abstract methods. With interfaces you can implement the functionality of class defined with the interface keyword. Actually, you can implement from multiple interfaces if you need to. You can only inherit from one source, but implement many. The java documentation explains all of this in a easy to understand manner, see the article on the abstract keyword.

Q. Can an abstract class have a default behavior?
A. Yes, an abstract class may have normal methods.

Q. Can an interface have a default behavior?
A. No, an interface only has abstract methods and an abstract method has no body.

Interview Questions from a Start-Up

2009-07-21T19:37:00.000-07:00

I have also recently had a phone-screen with a Start-Up located in the Boston metropolitan area. Here are the questions that were asked for a Python/Java related programmer position. Maybe when you are asked these questions, you'll have a better response! :-)

PYTHON
Q. What is the syntax of a lambda function and why would you use it?
A. It starts off with the lambda keyword followed by argument(s), a colon, and a statement(s). A very simple lambda to return the square of an element may look like: lambda x: x*x

Lambdas are commonly use to apply a function across a set of data. Using it will allow you to use a function without actually defining one.

Q. What's the difference between a list and a tuple?
A. A tuple is an immutable data set, where the data will not change. A list is a collection of elements whose size varies with the data it contains. A tuple is typically faster and takes up less memory compared to an identical list.

Q. Explain the difference between calling .items or .iteritems on a dictionary? How about the functions range or xrange?
A. Calling items() will return a list of all key-value elements in the dictionary. The other method, iteritems(), will return an interator object and can be more memory efficient when working with large data sets. The xrange() method is similar to iteritems() as it provides a similar iterator when defining a numerical range: i.e. 1 to 1,000,000.

Q. What is the metaclass?
A. From what I understand, it allows you to modify the definition of a class at run-time. It's typically used as a "factory" or a template for creating classes. In some examples, it seems as a way to extend a class, providing it some of the same functionality of another class. The write-up at voidspace.uk.org helped a bit when trying to understand some of the black magic that I've seen a few times in the past. Sigh, I do not have much experience here.

Q. What are list comprehensions?
A. Example: [person.name for person in people_list] will return a list of names. Basically, list comprehension is an efficient way of creating a new list (it just is!) without using the typical for-loop construct, or a map() or lambda function.

JAVA
Q. What is the difference between an int and a Integer?
A. An int is a primitive type and the value is stored in memory. An Integer is the reference type, or Object, form of the int primitive. The value of this Object is points to an address and does not contain the value of the integer. You would use Integer when working with collections or other various methods that require an Object rather than a specified primitive type.

Q. What is the difference between the == and dot-equals operator? (Actually, I didn't understand what the interviewer was asking and I thought he meant a .= operator, not the .equals method... whoops!)
A. The double equals sign represents equality between two objects. This operator checks whether or not two addresses are the same. The .equals() operator is typically an overrideable method where you define which variables will be compared to determine the equality of two objects. By default, it is similar to the == operator.

Q. Explain a String, the StringBuffer, and StringBuilder and how they differ.
A. A String is a reference type that represents a set of characters. Strings combined together will create new instances in memory as a String itself is immutable. The StringBuffer and StringBuilder Objects are common utilities to use to improve the efficiency of concatenating strings. The difference between the two is StringBuffer is synchronized and StringBuilder is not. Thus, StringBuilder is typically faster.

Interview Questions from a Mid-Sized Company

2009-07-21T18:00:00.000-07:00

I've recently interviewed at a mid-sized company, probably best classified as a "Start-Up." This took place near the Boston metropolitan area. Unfortunately, I may have not answered all of the questions correctly or coherently but I'm certain someone will appreciate this in the future. I've provided a very simple and general response to most and I suggest you do a little more research.

Below are the Java, JS/CSS, and Database questions that I was able to remember. The company had a little more than 50 employees and they were looking for a UI Java Developer.

JAVA
Q. What does the keyword synchronized mean?
A. That a particular field of an object may only be accessed by one thread at a time.

Q. Can you describe polymorphism?
A. I gave a convoluted answer and the interviewer was actually just looking for "Having multiple forms."

Q. How do you create a thread?
A. You can inherit the Thread class or implement Runnable. You would have to implement Runnable if your class is already extending another class. Java only supports single-inheritance. This question is very common and a good resource is at exampledepot.com.

Q. What will happen with the following class when you have several threads accessing both methods at a particular instance in time?

class Thing {
void doThis() {
System.out.println("Hello");
}
void synchronized doThat() {
System.out.println("Goodbye");
}
}

A. Each thread that accesses doThis will cause the statement "Hello" to be printed. However, "Goodbye" will only be printed once as every other thread will have to wait until a lock a released on this method. Synchronize will only allow this method to be accessed by one thread at a time.

JAVASCRIPT/CSS
Q. What's the best way to concatenate or combine several strings?
A. Put the strings in an array and combine them with the join method. Example:

some_text = new Array("Python", "does","this.")
resulting_text = some_text.join(" ");
/* The result is "Python does this." */

Q. Why would you pass JSON instead of XML?
A. In general, parsing data in a JSON object within HTML/JS is typically much faster. A good discussion on "when to prefer JSON over XML" took place on StackOverflow. A user, Robert Rossney wrote:

Favor XML over JSON when any of these is true:

You need message validation
You're using XSLT
Your messages include a lot of marked-up text
You need to interoperate with environments that don't support JSON

Favor JSON over XML when all of these are true:

Messages don't need to be validated, or validating their deserialization is simple
You're not transforming messages, or transforming their deserialization is simple
Your messages are mostly data, not marked-up text
The messaging endpoints have good JSON tools

Q. What does a JSON object look like? How do you create one?
A. From what I know, it starts and ends with a curly brace. Within it are key-value pairings that describe the data.

{
"first_name": "Donnie",
"last_name": "Demuth",
"location": {
"city": "San Diego",
"state": "CA"
}
}

Q. How do you bind a function to a variable? And why would you do this?
A. Simply assign a function, as you would normally but with or without a name, to a variable. By doing so, you can assign functions to particular Objects or pass a function as an argument. Justin Palmer explains it in a real fun way at alternateidea.com. Here's a very simple example:

var my_obj = new Object();
my_obj.add = function(x, y) { return x + y }; 

/* The object has a function named add and you can use it by */
my_obj.add(1, 2);

Q. What is faster, $("p #dog") or $("#dog")?
A. The later is faster, $("#dog") because the first example would first have to find all "p" (paragraph) elements and then parse those elements for the "dog" key. You can simply search for that key immediately because all ids should be unique.

Q. What is faster, $("p div.dog") or $(".dog")?
A. Since a class can be used multiple times, the first would be faster. We first narrow down the search and then look for a particular element with the "dog" class-name.

DATABASE
Q. What's the difference between an inner and outer join?
A. An inner join returns a combined set of data where two tables share a common criteria. An outer join is a data set of all records from one table with data from the other where a common criteria is also met.

It's probably best to think about a two-circle Venn-Diagram. The center, or where the two circles overlap could be considered the results of an inner join. An outer join could be either circle and the part other the other where the two overlap.

Q. Describe the relationship between the following three tables: an Employee, an Application, and an Account Table.

Employee
int employee_id
String first_name
String last_name
String email

Application
int application_id
String app_name

Account
int account_id
int employee_id
int application_id

A. The Employee and Application tables do not have any foreign relationships. The Account table maps the two tables together, defining which users have access to an Application. The Account table would probably not have unique restraints on the employee_id or application_id columns.

Q. Write a query that returns the email for all users that can access a particular Application. Let's say, an Application with the name of "Sales".
A. Note: There's probably a better way to do this. Here's an excellent write up on multiple sql joins.

SELECT Employee.email FROM Employee
INNER JOIN Account ON
Employee.employee_id = Account.employee_id
INNER JOIN Application ON
Account.application_id = Application.application_id
WHERE
Application.app_name = "Sales";

Q. How would you map these tables to Java objects and what would those objects look like?
A. I would probably create Pojos (plain old java objects) that map directly to these tables.

class Employee {
int id;
String first_name;
String last_name;
Email email; // use a specific Email class for emails
}

class Application {
int id;
String name;
}

class Account {
int id;
Employee employee;
Application application;
}

Setting up PostgreSQL on a Mac (for Django)

2009-07-12T12:15:00.000-07:00

Due to my infatuation with the google-app-engine and aep I feel like I haven't set up Django with PostgreSQL in quite a while. In this post, I'll go over the few simple steps you can choose to follow to get postgres up and running on a Mac. If you haven't done so already, please install pip.

POSTGRESQL INSTALLATION
Okay, so there's many different ways to install PostgreSQL but I'll go over the point-and-click method a lot of you are used to. First you need to find the thingy to click on so download the PostgreSQL dmg (installation file). At the time of this writing the version is 8.04. After downloading the file, give it a good double-click. A dialog window will pop up enabling you to click on the postgresql-8.4.0-1-osx icon. Give that a good click and follow it up with clicking on "Open" and entering your password.

There's a good probably that you'll be prompted to reboot your computer before the installation continues. If so, please reboot and double click on the dmg after your system starts up again. Following the installation setup, you will be asked to enter the following:

Enter a installation directory: /Library/PostgreSQL/8.4 (default)
Enter a data directory: /Library/PostgreSQL/8.4/data (default)
Enter a superuser password: kryptonite (can be anything)
Default port 5432

You may be asked if you'd like StackBuilder to run after the installation completes. Say yes.

StackBuilder will allow you to install a few goodies such as phpPgAdmin to help with DB administration.

Depending on what you clicked on, you may have to go through a few installing procedures. Just click on the next button until all those pop-up windows disappear.

SETTING UP THE REST
We'll have to add a few items to our environment so open up a terminal window and enter the following:

donnie-demuths-macbook-air$ echo "export PATH=$PATH:/Library/PostgreSQL/8.4/bin" >> .bash_profile
donnie-demuths-macbook-air$ echo "export PGDATA=/Library/PostgreSQL/8.4/data" >> .bash_profile

Those two commands will modify your system's PATH variable and add a new one called PGDATA. After running both, close and reopen a terminal window. You can now use a tool called psql to work with your database in a shell.

Before you use psql, let's start the server via the easy way. Click on your finder and go to Applications > PostgreSQL 8.4 > and double-click on the Start Server button. There you'll also find a Stop Server button when you need it. Obviously, there's way to control your db via the command line but it's pretty easy to forget those commands/scripts.

Once the server is started, open up a terminal and type:

psql -U postgres

To enter the postgres shell with the postgres default user. You'll be prompted to enter in a password, mine was "kryptonite."

If you want to use PostgreSQL with Django, you'll need the psycop2 python package. Install it with the following command:

donnie-demuths-macbook-air$ sudo pip install psycopg2

To install Django you can also use the pip utility:

donnie-demuths-macbook-air$ sudo pip install django

Be sure to add django-admin.py to your bin by typing:

donnie-demuths-macbook-air$ sudo ln -s /Library/Python/2.5/site-packages/Django-1.0.2_final-py2.5.egg/django/bin/django-admin.py /usr/local/bin/django-admin.py

Full Django support on the GAE, 1.2.3?

2009-06-19T08:33:00.000-07:00

Gosh... It looks like I have a lot of porting to do.

Google just released the 1.2.3 version of the App Engine which includes full support for Django 1.0.2. What this means is that I can now ditch the Google App Engine Patch (a collection of hacks to get Django running on the G.A.E) for one of my current sites, www.aboutaplace.com.

I am extremely excited to test this out.

Btw, the Google Maps version 3 is completely amazing. I've recently developed tools for it on www.signonsandiego.com and www.aboutaplace.com.

Naked Domains for Google App Engine, (WWW)ahoo!

2009-05-28T08:35:00.000-07:00

After several months of frustration, I can finally assign a Google App Engine project a www domain easily. No more URL Frames *shudders* or Redirects! Cheers!

In the past, when trying to add an AppEngine project to the www domain, I would be prompted with a conflict: Sorry, the www sub-domain is taken. Please choose a different sub-domain for your project. Actually, the error message was a bit more USELESS than that but if you ever saw it, you'd know what I mean.

Well, guess what? It was taken. By google-sites!

To free your domain of the www hi-jacking, follow these 3 easy steps:

(1)

First, log into your google apps account by going to http://www.google.com/a/. You'll need to click on the Returning user, sign in here on the right hand side to enter your credentials.

(2)

Click on the Service Settings tab on the far right of the tool bar. And then click on Sites.

(3)

Here you'll find a little bugger of a tab named Web address mapping. Click on it! That's where the evil pirates are hiding... After doing so, you'll see that www.mywebdomain.com was pwn3d by google-sites!

Delete! delete! delete!!!+shift1!!

The Bear Minimum - Decision Making, The If Statement

2009-04-15T06:15:00.000-07:00

What would you do if somebody asked you the question:
Are you ready for some football?!

If you like football, I'm sure you would say something like "I can't wait, I looOOove the {{insert Team Name here}}!". Otherwise you'd say "Nope" to some degree.

An expression like this can easily be represented in code. A super simple program may be represented as:

Prompt a message "Are you ready for some football?"
If the user types in "Yes" the display "I can't wait!"
otherwise display "No way!"

The general form of making a decision in a programming language is with the if-statement. A single if-statement performs a task similarly to if something is true then do this. Additionally, the if-statement has a best friend -- the else keyword. In most languages you can follow the if-statement with an else to do if something is true then do this else do that.

Java Examples:

boolean cond = true;
if (cond) System.out.println("The condition is true!");

boolean conditionTwo = false;
if (conditionTwo) System.out.println("The second condition is true!");
else System.out.println("The second condition is false!");

The above will print out "The condition is true!" and "The second condition is false!" When you've mastered the art of using if-else, you can also pick up another trick -- the else-if-statement. This allows you to chain together multiple if statements where only one of the statements will ever be executed.

Java
Here's a simple program named DecisionMaker.java. This program attempts to compare an integer with various conditions.

class DecisionMaker {
public static void main(String args[]) {
    int i = 0;
    if (i == 1) System.out.println("one!");
    else if (i == 2) System.out.println("two!");
    else if (i == 3) System.out.println("three!");
    else System.out.println("The variable i is not one, two, or three!");
}
}

The output will be "The variable i is not one, two, or three!"

JavaScript
In a file named DecisionMaker.html

<script type="text/javascript">
i = 0
if (i == 1) document.write("one!")
else if (i == 2) document.write("two!")
else if (i == 3) document.write("three!")
else document.write("The variable i is not one, two, or three!")
</script>

The output will be "The variable i is not one, two, or three!"

Python
In a file named DecisionMaker.py

i = 0
if i == 1: print "one!"
elif i == 2: print "two!"
elif i == 3: print "three!"
else: print "The variable i is not one, two, or three!"

The output will be "The variable i is not one, two, or three!"

The Bear Minimum - Variables and Simple Arithmetic

2009-04-09T18:15:00.000-07:00

In order to program, you need to be familiar with some basic algebra concepts even though you don't need to be any decent at math. The following should make sense to you and you should be able to tell me the answer in every situation:

a = 1
b = 2
c = a + b + 1
What is the value of c?

m = 3
n = m - c
What is the value of n?

x = 5
y = x * 2   (where the asterisk is the multiplication symbol)
What is the value of y?

With algebra we are introduced to the concept of variables. A variable, as daunting as it may sound, is simply some 'thing' that contains 'something' else. Above we have several variables: a, b, c, m, n, x, and y and they will store a number. In these examples, these numbers are whole numbers -- oh that's just an observation for now.

Going through this from the top to the bottom we can conclude that:

c = 4
n = -1
and y = 10

But that took brain power to solve. Let's get the computer to perform the calculations.

Java
In order to create a variable in Java we need to specify what type of 'thing' it is going to contain. One of the types you can use is an Integer or int. An int represents a smaller whole number. You should use it for a variable that you know will never be greater than a trillion. To declare what type a variable is going to be write it before the variable name. If you want the variable "a" to be an "int" you can write it as "int a".

Here's the code and make sure that you save it as SimpleArithmetic.java:

class SimpleArithmetic {
    public static void main(String[] args) {
          int a = 1;
          int b = 2;
          int c = a + b + 1;
          System.out.println("What is the value of c? " + c);
          int m = 3;
          int n = m - c;
          System.out.println("What is the value of n? " + n);
          int x = 5;
          int y = x * 2;
          System.out.println("What is the value of y? " + y);
    }
}

Compile and run the java file and you'll get he desired results.

JavaScript
JavaScript and Python is a little more 'loose' than Java where you don't need to specify the type of your variables. The same code in JavaScript should look very similar to the algebraic expressions. Also, in this example and printing out three "break" tags in order to print the messages on separate lines so I hope that doesn't confuse anyone.

<script type="text/javascript">
a = 1
b = 2
c = a + b + 1
document.write("What is the value of c? " + c)
document.write("<br />");
m = 3
n = m - c
document.write("What is the value of n? " + n)
document.write("<br />");
x = 5
y = x * 2
document.write("What is the value of y? " + y)
document.write("<br />");
</script>

Open up the file in your favorite browser and you'll be greeted with the following:

Python
Like JavaScript, this is a loosly-typed language. You don't have to specify what the type of your variables. In this file, named SimpleArithmetic.py -- BTW unlike Java, Python files aren't coupled to the name of the class -- we can write code that look eerily easy to understand.

a = 1
b = 2
c = a + b + 1
print "What is the value of c? %s" % c
m = 3
n = m - c
print "What is the value of n? %s" % n
x = 5
y = x * 2
print "What is the value of y? %s" % y

Doesn't python look beautiful? Executing the code is easy as pie as well.

The Bear Minimum - Simple programs using "Hello World!"

2009-04-09T05:26:00.000-07:00

Whenever you try learning a new program language, I guarantee that you'll run into the phrase "Hello World," among others like "Foo Bar." HelloWorld is a simple application that prints out a phrase to the user and provides an experienced programmer the syntax and grammar on how to use this language. In some ways, its referred to the simplest application you can write for a programming language.

Below we'll create three different HelloWorld programs.

Java
In order to create this program in Java we'll have to create a file named HelloWorld.java. Please use a text-editor such as NotePad or Vi to do so. Within this file, we'll write out some java code that will allow us to print the message to the user.

Here are the contents:

class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hello World!");
    }
}

Before you can run this file, you will have to compile it first. Compiling takes the code that you've written and converts it into a language that the machine (Java Virtual Machine) can understand. Before you compile, go to the directory that contains the file with your command-line tool. If you saved the file under "c:/myprograms" you can type "cd c:/myprograms". Make sure that the file exists and has the ".java" extension. You can do this with the "dir" command.

To compile type:

javac HelloWorld.java

If your code is valid, it will create a file named HelloWorld.class and not prompt you with any errors.

To run the program type:

java HelloWorld

Notice that when compiling you must pass it the filename and when you try to run the program you only use the name of the class.

JavaScript
To create a HelloWorld app in JavaScript, let's create a new file named HelloWorld.html. Within this file use the following:

<script type="text/javascript">
document.write("Hello World!");
</script>

Make sure that the editor that you're using saves the html file without adding any extra markup. Some will manipulate the characters above in order to make it "readable" to humans rather than executing it as code.

To run this code, open HelloWorld.html in any web browser such as Internet Explorer or FireFox.

Python
In a file named "HelloWorld.py" type:

print "HelloWorld!"

Then on the command-line, within the same directory as HelloWorld.py, run the code with the following command:

python HelloWorld.py

Easy ain't it?!

The Bear Minimum - Finding your path

2009-04-01T23:54:00.000-07:00

Since most people are Windows users, you will probably have to modify your path variable to make programming a little easier for you. If you're using Ubuntu or OSX this should already be set up for your or you'll already know how to do it!

The path variable allows you to access files and programs easily from any directory or folder on your computer. For the most part, you will be compiling and running Java programs from the command-prompt so making sure your path correctly is extremely helpful. Before you modify your path variable, first you'll need to know where the JDK was installed -- it generally installs to c:\Program Files\Java\jdk1.6.0_13 where these numbers at the end specify the version.

Make sure that the jdk directory exists and you may want to look within the bin folder. Within it you should see a file named "javac.exe".

We want to be able to easily access the javac program from anywhere on the command-prompt so we'll want to add that bin directory to our System's path. Based on your flavor of Windows, finding the path variable may be a journey of it's own. You should be able to find it by following:

Start > Control Panel > System > Advanced > Environment > Variables
OR
Start > Control Panel > System > Advanced Settings > Change Settings > Advanced > Environment > Variables

Basically we need to modify the environment variables. Once you find it, there should be two lists: one for user variables and one for system variables. Let's change the system variable as it will make the change for all users and programs. Within the system variables list, find the word "path" under the variable column and then click on edit.

And carefully add the following line to the front (or to the left of the existing text):

c:\Program Files\Java\jdk1.6.0_13\bin;

Remember, if your jdk directory exists at a different location on your computer, please make sure you use it! Your jdk and bin directory may be at c:\Program Files\Java\jdk1.5.0\bin or c:\java\current_jdk\bin so make sure you found it first.

And yes, there was a semi-colon.

JavaScript
You will not have to add anything to your path.

Python
You may need to add the Python folder to your path. Find out where you installed python and make sure that the directory has the python.exe file. It may be at c:\python26 or c:\python.

Follow the same steps from above and add the following to your path variable:

c:\python;

Don't forget the semi-colon.

The Bear Minimum - Finding and Installing the required software

2009-04-01T22:00:00.000-07:00

In order to write code all you really need is a keyboard and a text-editor. You don't need anything special at all! You can even write code with a pen and paper, but I doubt a computer would ever make sense of it.

The code is meaningless without the tools you need to compile and run it. For Java, these tools are packaged in the Java Standard Edition (SE) Software Developers Kit (SDK) -- yes I know, it's a long name. Navigating http://java.sun.com is pretty complicated so here are some images to help.

On the right rail, you'll see a list of Popular Downloads, click on Java SE. Try to fight the temptation of clicking on Java EE 5 SDK!

Then on the downloads page, click on the Download button that follows Java SE Development Kit (JDK). And again, avoid the temptation of clicking on the first button, Java SE Runtime Environment (JRE). The JRE allows you to run java programs but not compile or "make" them.

Finally, on the next page fill out the required information to begin the download. After it's completed, install it!

Javascript
JavaScript is a programming language that you probably already have. It's packaged with most modern browsers. Congrats! You don't have anything extra to do here... Except that if you're using Internet Explorer you may want to consider downloading FireFox or Opera.

Python
Python is sometimes considered as a lowly-scripting-language but it's a very powerful and modern language that I highly recommend. In order to install this language go to http://python.org and click on the download link on the left rail. You'll want to download a version of Python 2.6+ because the next version, Python 3+ is not fully adopted yet.

The Bear Minimum - Introduction

2009-04-01T21:00:00.000-07:00

The Java programming language can be daunting to learn. Especially for you, someone with little to no programming experience! Learning a programming language is like learning a spoken language such as Spanish or Mandarin Chinese -- but probably much easier!

But hopefully by the end of the class you would have picked up some of these programming basics and feel a little more comfortable with Java. Many programmers (see PHP/Ruby programmers) complain about how HARD Java is, so consider learning this a real accomplishment.

Within this series of guides I'll cover some of the "bare minimum" topics that you really should memorize. I'll also show you how to do the same things with JavaScript and Python -- although consider that an optional and additional lesson. At the end of the nine weeks, I'll package everything up into a pdf and send it out to you all.

Oh an I'm calling this guide "The Bear Minimum" because of the following Yahoo question and answer:

Programming on a MacBook Air

2009-02-15T21:38:00.000-08:00

I program. I plan systems. I build them. I design them. So why aren't I using a MacBook Pro?

The main reason for this is that I'm frugal and that I'm a coffee house junky. When I noticed that MacBook Airs were finally on the refurb list at the Apple Store, I had to jump on one. For $1299, I purchased an Air with 1.8Ghz and a 64gb SSD Hard-drive. If I bought this last year, it would have easily ran me over $3,000.

My biggest concern with this purchase is how it would hold up. Could I bare the 13 inch screen? Can I transition from a powerful and clunky Thinkpad to a stick thin sheet of aluminum?

I scoured the web for answers. I wanted to know whether or not someone is actually using the Air as a rig for programming. If you tried performing this query, you may find this thread from google:
http://forums.macrumors.com/showthread.php?t=635322

Unfortunately, no one really provides any answers in it. I would have to find the answer out for myself.

Before we look at the Air, let me first tell you about my previous rigs.
- MacPro Desktop w/ 23" Cinema Display @ 1920 by 1200
- 15.4" Lenovo Thinkpad T61p @ 1440 by 900
- 17" Dell XPS fully upgraded @ 1920 by 1200

Those are all pretty powerful machines. Probably in most cases, more powerful than I need. What strikes me the most are the resolutions. I'll be losing from 400-200 pixels on the laptops. However, the T61p and XPS are Windows XP boxes and this operation system has been a thorn in my side lately.

With my Air, I'm running a resolution of 1280 by 800. Due to it's size (13"), it probably wouldn't make any sense to run it at a higher resolution. After playing around with the system for a bit, it's noticeably not as smooth as the MacPro -- obviously. Setting up my environment was a breeze though, compared to getting the same thing running on a Windows box.

I'm currently working on an Python gig where I'm using Django, the Google App-Engine, and TextMate as my IDE. I prefer TextMate over PyDev on Eclipse because it's light weight, elegant, and I don't need to worry about project files.

Running the project, you'll need several windows open. A terminal window (for debugging and various shell tasks), a browser (to test the interface), and one for your IDE or TextMate. I've attached a few screen shots of my development environment for those who need to see what it may look like.

With "Spaces" from OSX 10.5 or Leopard, keeping all windows open is very pleasant. Switching between windows can be performed easily using the control button plus an arrow key. It's much easier than switching between windows using the alt-tab method. This is very close the programming experience you'll get with using two monitors. I'm SOOO glad I have it on the MacBook Air. Without Spaces, I'm not sure if I would find my self gushing over this laptop.

So yes, I am gushing. I love the experience so far. Spaces, TextMate, Terminal is everything that I could ask for in a programming rig. Although I guess I could just boot into Ubuntu on my Thinkpad and get a similar experience. However, I do fairly often boot into Windows for Word and PhotoShop... Sorry Open-Office and Gimp, it didn't always work out between us.

But this little Air should be able to handle Office and Photoshop. I haven't tested it out yet but I'll post my thoughts when I do. Bottom line, I wanted a Mac and I think I purchased the best one I could under $1500 dollars. Would I pay $2500 for an Air? Probably not, I think I would get a Pro if I had the extra money. Do I regret not shelling out an extra grand for the Pro? Heck no. The Air does everything that I need.