Machine Level Programming I. Software Architecture: Fill & Download for Free

Because it is not software development that is hard, it’s the fact that not enough people have adequate quantitative education to pick up software development, let alone Computer Science.As someone who started and managed educational technology companies, I would say beware of all the hype train surrounding CS education.Here’s why:As far as software engineering goes, I was self-taught before college. No one in my family knows how to properly use a computer (and now they’re even struggling with an iPad), but I picked up a book and taught myself how to program by just spending countless hours reading and poking around with my 586.When I started I was 12, there were concepts that I couldn’t understand until a year or two later, simply because I haven’t learned higher level algebraic or logical concepts at my age that I couldn’t grasp certain logical relationships between software components. The concepts and relationships critical to software development are actually already taught in schools, but the problem is that not enough people are even finishing schools.I’ve been working in the field of AI for more than a decade, and I’ve been asked by students “why do I need formal education? I just need to learn software engineering” and my answer is always the same: Because real-world problems are complex, for example I’ve worked on cognitive models for education, this require an intimate understanding of how humans acquire, store and process information, and to employ machine learning to analyze behaviors at a commercial scale, you need to know your linear algebra and statistics to design something worthwhile. Similarly when I start to work with natural language processing, you can’t do any serious work without spending enough time understanding how humans handle words, sentences and conversations - and more often than not, there is logic involved, there are statistics involved, there are heuristics involved. You need to learn all of this, one way or the other, the to be honest, formal education is the eastiest way to do it.After high school I went to Carnegie Mellon for computer science, as some people may know CMU is one of the most pretigious schools for the discipline. And truth to be told, pretigious computer science schools like CMU, Stanford, MIT, Berkeley, etc are not educating students about computer science - these schools are playgrounds for smart and well-educated folks to grow, most of the time in a self-directed fashion.Why do I say that?Because these schools generally have incredibly smart and talented professors who don’t really like teaching undergrads (or just don’t like teaching at all, period), they don’t have a patience for it, in fact, most of them don’t really even know how to teach.These schools are not turning average joes into computer scientists, make no mistake, they are taking the highest achievers from high schools, and then give them a sh*t ton of work and opportunities, and see who survive the gauntlet.When you fail at these schools, there is little to no support to get you back up and running - you have to stand up, dust it off and keep going. You have to push yourself to get back into the game. Because you’re constantly surrounded by people who’re well-versed in many different disciplines, who’re creative and who’re super competitive.In other words, there is no shortcut for simply getting better educated.So why is teaching software architecture to kids a bad idea?It’s simple, why would you teach a kid who can barely grasp the concept of static equilibrium, something as specialized as carpentry?Software architecture and software engineering are the same way. These professional skills are constant shifting targets. 25 years ago, Windows and desktop apps were all the rave. 15 years ago, it was all about browser-based software. 10 years ago, all the architecture and practices were about mobile apps and social network content. And now? things are shifting to messaging and chat.What if Quantum computing becomes mainstream? holy moly, now much of what you learned in computer science and software engineering before may have become irrelevant.A computer science graduate from Berkeley will survive all of this, because they were educated well enough to apply general analytical and reasoning skills to acquire new professional skills.But for a kid? Every five years what you’ve learned is a complete waste of time, if you never learned broader analytical and reasoning skills. A person’s time is better off invested in these than in learning software engineering at a young age.Oh, lastly - we are not born with logic. We have some logical intuitions.Logical inferences still need to be taught. Our innate intuitions are not enough to grasp computer science and/or software engineering.

I don’t know how representative of the typical experience my answer will be.As a bit of background, I graduated with a few distinctions from the BS program with a minor in applied mathematics, experience at a summer school for PhDs, and research experience at the CUPLV lab for programming languages and verification that resulted in a publication. My curriculum and schedule was designed to be more demanding than is typical for this program (though there were certainly other undergrads in my program doing similar, and getting a minor or double major seemed anecdotally common). The program is generally structured with the aim that different students wanting different things out of their studies can get what they’re after, with mixed results.I’ve not taken other undergrad CS courses, but our core curriculum seems comparable to most flagship state school programs in the US and Canada, with an emphasis on drilling career-relevant fundamentals, project work (some group-based), and making software technologies applicable to daily life and society.Our Computer Systems course, CSCI 2400, tracks the corresponding CMU course, at a slightly slower pace with instruction in CPU architecture added. Since this course is based on one of the most demanding required courses at a school with a reputation for an intense CS curriculum, it’s one of the toughest required course we have in terms of demands for study and work and is likely designed to filter out students unprepared or unsuited for computer science. Programming Languages is also a quite demanding class work-wise and has a reputation for varying considerably in quality with the instructor. Similarly, our algorithms class veers a bit to the hard side of average.Some optional courses, such as Operating Systems, were also tough in a similar way (although the undergrad version of this course doesn’t entail writing a kernel as it does at a few schools). Some of my coursework, like in Databases, felt comparatively softball. Despite skewing my coursework towards theory, I was a bit disappointed that courses like AI didn’t involve more theory.Demand for our program driven by economics has put serious logistical stress on it. This unfortunately means some courses are taught by temps who are given overwhelming workloads or (more a problem with theoretical than hands-on courses) don’t entirely understand the goals of the faculty who designed particular courses. Quality often suffers as a result. If possible, take anything other than industrial/real-world practica with tenured or tenure-track faculty. With that said, I know people who took Boulder’s engineering coursework in high school concurrent programs said they found the quality of instruction and content generally higher at Boulder than at their Ivy League undergrad engineering programs. Make of this what you will.On a core course basis, our applied math program feels noticeably more intellectually challenging and time-consuming. I grew considerably as a result of it. Some newer classes that overlap between CS and mathematics, like Machine Learning and Operations Research, benefit from the rigor.I actually think highly of our humanities instruction (while the humanities student body varies a lot in seriousness), and if you have interest in a particular humanities topic, it would make a good minor.By far the best CS coursework I took was upper-division or graduate-level in narrow topics that our faculty specialized in, notably Chaotic Dynamics and (grad-level) Program Analysis. Not only did I learn a great deal from these classes, but the instructors’ passion and depth of expertise really shone through and rubbed off on me. I wish our core curriculum benefited more from this kind of energy.As with most state schools, there’s a pretty wide variety of students attending the program, all of whom were at least above-average students in high school (the non-conscientious students simply flunk the program, I think deservedly given its goals). There’s a crowd of overachievers who exceed the baseline whenever possible and exude an almost manic energy that reminds me of students at hyperselective schools (some of whom turned such programs down due to cost). There’s also a far more laid back crowd. Compared with typical CS students, our undergrads seem either very career-minded; broish/fratty in a sort of non-verbal way; hippie-like and granola-ish, with corresponding skepticism of capital or modern society (though generally lacking the radical/avowedly Marxist-materialist edge of schools like Berkeley); or passionate about forms of media and experience enabled by CS topics like video games and HCI. There are also a few theoreticians. There’s a veneer of modesty, friendliness and welcoming energy that students tend to exude (sometimes superficially and sometimes quite genuinely). Cutthroat attitudes are usually hidden from sight, as are stress, loneliness and sadness, behind an affable and enlivened exterior. Overt self-aggrandizement is usually scoffed at. There’s a strong culture of collaboration, and tutoring is a common student job. Most other students were quite nice to me. I asked lots of questions in class, which I know for a fact came across as quite helpful to some and very irritating to others.With all of this in mind, I suggest the following to students entering the program:As a unifying theme, have a specific plan for what you aim to get out of the program and orient things around that.Explicitly delegate time for preparation, study, and assignments. Even doing the minimum necessary, CS is relatively rigorous and doing well in the program isn’t possible without a regime around mastering the material. Practice tough material regularly by doing exercises, learning concepts and doing counterfactual exercises (eg: “What if x, y or z were different? How would that change this situation?”).If you manage your time well, are a fast study, and have passions outside CS, I recommend a minor or double major. Math (pure and applied), physics, linguistics, economics, and various foreign languages are common choices for this. But if you need more time for the material, have other commitments, or want comfortable margins of unstructured time, best to focus on CS.If you’re passionate about a narrow area of CS or want to continue your academic studies after undergrad, consider doing research. If you pick the right lab and work hard, you’ll work with and be mentored by world-class faculty alongside excellent researchers on interesting projects, could get an expenses-paid trip to a conference or NSF or DARPA workshop, and could walk away with pyrophorically glowing letters of rec. We seem to have good Systems, NLP, PL, and HCI labs (CSRankings considers us 9th, 16th, and 22th in the world for HCI, logic/verification, and PL publications in top-tier journals, respectively, beating out Ivies, Chicago, MIT, and Stanford for some of these).Get friendly with tenured and tenure-track faculty. Ask them for help and engage with them concerning both the classes you’re in and their work. This will help your performance in class, your long-term understanding of the CS topics they specialize in, and your career (whether in the academy or industry).Try and meet a lot of your fellow students, with the joint goals of being on good terms with a decent number of other people in your program and finding a small community of friends you can relate strongly to within the larger community. This is actually quite important to studying well and weathering stressors and misfortunes.Truly think about and invest in picking the right courses. Get the balance of courses right (do not exceed five per semester unless you absolutely must). Don’t take a heavy load for its own sake. Carefully pick courses based on FCQ and professor reputation, and try and take courses with professors instead of adjuncts.Don’t take courses out of sequence (this burnt me a little sophomore year).Pick out-of-school activities carefully according to passions and psycho-emotional needs. It’s better to spend time doing a few things that are important to you rather than a dozen things superficially.

What are Cobol, Fortran, and Pascal?Short answer, the first two are two of the primordial so-called production ‘high level’ programming languages from the 50s and 60s, and the latter is a language designed to teach programmers from the mid to late 70s.Longer form:First COBOL: Formed in the late 1950s, the “Conference/Committee On DAta SYstems Languages” or CODASYL was originally created to try to develop a standard programming language for business [logic]. They created a new language [COmmon Business Oriented Language or COBOL] and a number of standards along with what would later be called the CODASYL Data Model.This work predated what eventually become ‘Relational Data Model’ which is what is used by most modern databases. Because early business systems were developed largely “in-house,” teams of COBOL programmers were often found at financial services firms to develop and maintain the programs and databases used to ‘run the business’ originally on large so-called ‘mainframe’ systems such as those developed by IBM and the “BUNCH” companies. Because their in-house programs were custom creations, the firms that were heavily invested in them tended to be well financed.By the mid to late 1970s arrived a few important things that change the rules: first, IBM invented System R and its relational calculus; along with the first versions of the Structured Query Language or SQL. Next, the type of system that Gordon Bell called a ‘minimal computer’ or ‘mini’ (minicomputer) started to appear that were large enough to ‘self-host’ a meaningful database. Finally, other firms such as Oracle developed a knock-off relational database systems that ran on the mainframes as well as DEC’s VAX, DG and the like, and other firms such as SAP and BAAN developed commercially-off-the-shelf (COTS) business applications that could be custom configured for end users that called back into Oracle or other relational DBs.All of these actions enabled two other pieces of history. Mid-size firms could now start to have their own business applications without needing teams of large in-house programmers. Originally these commercially available products were not as good as the custom systems, but by a process described by Harvard Business School’s Prof Clay Christiansen in his book “The Innovator’s Dilemma”, this new scheme ‘disrupted’ the older CODASYL shops.While COBOL would add relational logic and calls into SQL, as a result of being able to purchase COTS application suites, use of it fell from favor and is less important as fewer and few production applications were developed or maintained.Next FORTRAN: In the late 1940s and early 1950s, large computers were mostly being used the same way as they are today since both are supercomputers of the time. Supers, such as those described in today’s Top500 such as the Lenovo Built SuperMuc, or the IBM 701 from the 1950s, are both basically running code that describes complex mathematical formulas. In approximately 1953, John Backus noted that writing those codes in assembly language was difficult (actually he is said to claim he was lazy). Backus proposed a ‘mathematical formula translation’ scheme, which by the mid-1950s would eventually be called FORmula TRANslation or FORTRAN.FORTRAN quickly became the ‘lingua-franca’ of the industry, in particular, the preferred programming language used as Wikipedia says:“Originally developed by IBM in the 1950s for scientific and engineering applications, FORTRAN came to dominate this area of programming early on and has been in continuous use for over half a century in computationally intensive areas such as numerical weather prediction, finite element analysis, computational fluid dynamics, computational physics, crystallography and computational chemistry. It is a popular language for high-performance computing and is used for programs that benchmark and rank the world's fastest supercomputers.”The point is unlike, COBOL, FORTRAN is still very relevant today; see my answer: Why is Fortran Still in Use? for the detailed answer, but the short form is: the work (i.e. the math) has not changed and the old adage comes to apply: “If it ain’t broke, don’t fix it.” And most importantly, history has shown that it has never been economically interesting to bother (or at least so far).Put another way, a good friend of mine from college used to be the chief metallurgist for the US Dept of Commerce’s NIST. As he points out, there are as many of 70 years of data sets that are known and have been analyzed with FORTRAN applications. If you magically could create new programs in ‘modern’ languages, those data sets would have to be re-analyzed. As I like to say, it is like the QWERTY keyboard, that train has left the station and it is not economical to redo it.Finally Pascal: As a response to some perceived issues with FORTRAN, a language was developed originally in the 1950s called ALGOrithmic Language or ALGOL, which is a bit like the story of System R vs. Codasyl, Algol was based on more mathematic principles than FORTRAN. It is interesting that the formal definition of the language, which was introduced with the ALGOL-60 report, the same designer of FORTRAN, John Backus, gave us the principle formal grammar notation for programming languages called the BNF or Backus-Naur form.As the 60s continued and there were different manufacturers of computer hardware, the need arose in the industry for languages to become defined in some ‘standard’ way; such that codes from one manufacturer had some chance of working on another manufacturer’s system. All of COBOL, FORTRAN and ALGOL ended up with standards committees, which is were engineers, mathematicians, and scientists that come together to define the syntax and semantics of each [FWIW: the latest Fortran 2018 standard was released last fall – see Fortran 2018 (Formerly Fortran 2015)].One of these groups would be International Federation for Information Processing (IFIP) an international, and (supposedly) apolitical organization whose ‘mission’ was to “encourage and assist in the development, exploitation, and application of Information Technology for the benefit of all people.”In the early 1960s, the IFIP Working Group 2.1 committee was created with the mandate to design a programming language to replace the then standard, Algol 60. The new definition was then being called ALGOL-X. At that time there were well-known deficiencies in ALGOL (much less the complaints about FORTRAN) such as the lack of support for a bit string data type.By the time of the regular committee meeting in the fall of 1966, three competing proposals had been brought forward. One of the proposals, by an academic named Niklaus Wirth, was for a language which offered some modest improvement over Algol 60. A much bolder proposal, by Adriaan Van Wijngaarden, was chosen by the committee that later became to be called ALGOL-68. So the ALGOL-X/ALGOL 68 effort took a different path and thus a whole new language was created, which was an interesting language, but it suffered the ‘second systems effect’ and was extremely complex and ended up being difficult to implement.As a result of the decision to pursue Van Wijngaarden's proposal, a number of the committee members, including Wirth and C.A.R. Hoare, abandoned the committee (it was said to have been a pretty intense meeting). Wirth developed his proposal into an Algol-style language which, eventually, came to be known as Algol W (like Algol 60 before it, the Algol W language's grammar was formally described using the BNF notation).Although less ambitious than Van Wijngaarden's proposal, Wirth's language introduced into Algol a number of new concepts including:a double precision floating point data typea complex data typea bit string data typedynamic data structures (i.e. what we now think of as structs and pointers to structs)block expressionsWirth was at Stanford at the time, and he defined and implemented a compiler for a new language, ALGOL-W for the IBM 360 architecture he had access, along with a new systems programming language he called PL/360. Once Wirth wrote the PL/360 compiler (IIRC ~1968), PL/360 was used to implement Algol-W to Stanford’s version of OS/360 and was ported to both TSS/360 at CMU, and MTS at UMich.Algol-W was (and still is) an excellent teaching language, but due to its short live era, I know of few books that exist for it. As far as I can tell no other systems really supported the ALGOL-W language until a UNIX implementation appeared a few years ago called AWE (which I have played with but not really used). FWIW, the original ALGOL-W implementation was one of the first ALGOL dialects that I used as a programmer, al biet at CMU under TSS.In the late 1960s, Wirth moved back to Zurich to become a Professor at ETH and took another stab at developing a teaching language. The new one was called Pascal. It was originally targeted for the CDC 6600 mainframe, which is the system described by himself and his co-author, Kathleen Jensen: “The Pascal User Manual and Report.” Wikipedia claims that the first successful implementation of the language was developed in a C-like language called Scallop by Max Engeli and was then hand translated into itself. The CDC compiler became the ‘root’ compiler of many different Pascal implementations that followed.My own first introduction to the language was on the PDP-10 using the compiler from H.H. Nagel and E. Kisicki, the compiler source of which can still be found at PDP-10 Pascal Compiler from DECUS Lib 10–06. If you look the source, you will notice that it was originally cross-compiled from the ETH compiler.The Pascal team at ETH wanted to propagate the language quickly. So, like our later developers who would create Java a few years later, the Pascal implementors developed a compiler “porting kit” that included a compiler that generated code for a “virtual” stack machine architecture, and an interpreter for that stack machine, instead of the CDC 6600 target. This took the name the “Pascal-P” system. One of the practical issues with the original porting kit was that the four versions of reference Pascal P machines (a.k.a. P1 - P4), assumed that the interpreter was running on a CDC 6600 with a 60-bit word length; when most of the hardware systems of the day used a 16, 32 or 36-bit word.None the less, the Pascal system did spread quickly. The first ‘byte code’ version of the P-machine was UCSD Pascal. By using the new version of the P-Machine interpreter, the UCSD team created a system that ran on a number of small (8-bit) systems such as Apple-II and later the IBM-PC.That said, other compilers (both ‘direct’ and ‘cross’ style) that generated direct machine code for most hardware ISAs of the day and was ported to numerous different operating systems. As a result, Pascal quickly appeared for different DEC systems, DG hardware, GE/Honeywell, much less the microprocessors such as the 8080/Z80, 6502, 68000, and the like.The fact is Pascal was a super teaching language and it appeared along with numerous implementations of itself at the right time. I personally think it is still one of the best/if not the best programming language for new programmers, but there are other languages such as Smalltalk that also vie for that distinction.I would highly recommend going to the Free Pascal web site and downloading the system and playing with it since the Free Pascal System can generate code for so many different systems and its likely to ‘just work’ on the computer that you are using the read this answer.Novice or expert alike, I believe that you can learn a great deal; particularly if you can dig up a used copy of one of the excellent books on the language and programming itself, such as Clancy and Copper’s ‘Oh Pascal’ and try the exercises contained within. Everything you learn about thinking how to program in general will map to today's languages such as C, C++, Java, Rust, or Go; and Pascal will give you a little bit of a safety harness which the others will not [like a say in other answers, we don’t put new pilots in a jet fighter, we put them in Piper Cub or a Cessna 150).But the popularity of Pascal and its original target as a teaching language did cause some huge issues, in practice. The original Pascal Report defined a language that was designed to be compiled as a student would: one complete program at a time. Fundamentally, the language lacked support for many features that ‘professional programmers’ needed. I will not try to describe them all here but rather point you to a paper by Brian Kernighan called: “Why Pascal is Not My Favorite Programming Language.” which he wrote after his experience rewriting his original FORTRAN based book: “The Software Tools” [ISBN 020103669X (ISBN13: 0785342036695)] to become “The Software Tools in Pascal.”Many of the issues Brian describes have been dealt with over the years. It was not that the problems with the language were fatal, as a lot of different Pascal implementations solved them; but alas in different ways. Thus program portability became an issue. Also, data representation became another practical issue. Strings, for instance, were defined as a containing a length stored in a length field before the string started [C does not keep a length in its strings, but rather an end-of-string marker]. Since most systems defined the Pascal string length in a byte and a byte as 8 bits, this means strings were limited to less than 256 chars; for those implementations. Other implementations (such as CDC, where a byte was defined as 6 bits, stored the length in a word), still others in a 16-bit word. Anyway, you can see where this is going … it made for code that was different between different Pascal implementations.The language was eventually standardized and most of the different extensions eventually came together. The commercial world created Object Pascal and Delphi for the IBM/PC which are OOP variants of Pascal (which Free Pascal supports). Wirth himself followed Pascal with a number of other languages, most notably Modula, then Modula II, Modula III and finally the Oberon family.All in all, Pascal had a great ride. Certainly, during the 70s and the 80s, there was a fierce debate between the C and Pascal worlds. The ‘pure CS types’ tended to prefer Pascal and the `Systems Programming’ types (like me) tended to prefer C. I’ve personally always thought the reason C++ gained such popularity with many in the CS community was that that way ‘C didn’t win.’Truth be known, I use what I need when I program. I’ve written way more C code than anything else in my 40+ year career, but Pascal or Fortran are fine tools. Fortran probably has more future just because there is so much code behind it and none of that is going anywhere, but Pascal (mostly in Delphi) still has a much smaller but still somewhat vibrant following. I’m not so sure about Cobol at this point, because most of the databases have moved from Codasyl and the custom Cobol code itself has been mostly replaced by commercial products, so it tends to not be economical to keep the old code going.Updated 04/04/19 to fix a couple of typos.