Why I run away from Knuth

May 8, 2008 at 5:54 pm | Posted in History, Programming | 18 Comments
Tags: , ,

someone named Greg commented on this post, very politely, that I am wrong in saying that Maths is not necessary. you can see his full comment in the comments after the post but I have quoted things here and have edited my original post to present a more fundamental answer to his questions.

Donald Knuth . Nearly every programmer, at some point of his life has got hit by this name or truly I should say got intimidated by his way of presenting algorithms. Many programmers think that he has wrongly titled his book as “The Art of Computer Programming”, though I don’t agree with them. Most programmers get scared when they look at his book. Heck, when, in 2002, I saw his book for 1st time, I thought, they were Mathematics books. Its only later when I started to do programming, in 2005 or 2006, I found out they were basically about algorithms. Still, I don’t read Knuth for some personal reasons which are totally different than what other programmers will state. These are all my personal thoughts about his style, I am not saying what Knuth is, all I am saying is what I feel about his writings and hence you can call it a biased opinion:

He is Quite academic. His books feel like they are written in a formal way, for schools or for MCAs, Post-Grads or Ph. D students. If you read these 4 books:

1.) How to Design Programs (HtDP)
2.) K&R2
3.) Practical Common Lisp
4.) Introduction to Algorithms

then you will come to know what I am talking about. All of these books have quite very different styles of presenting their concepts and the way they approach the problems are totally unrelated. As per my opinion, if you like K&R2, you will like Practical Common Lisp and if you like HtDP, you will never like K&R2 and Practical Common Lisp but you may like Introduction to Algorithms. HtDP is an academic book, written for students, I bought it from USA, for 6000 rupees and fro that time repenting why the heck I did the mistake. “The Little Schemer” has the same academic style. I am a real-life programmer, writing programs for some company and getting paid at the end of every month + 7 days πŸ˜‰ . I get frustrated at this academic style and by the same analogy, I also get frustrated when I see books I read in my Graduation, they were totally not related to what I am doing right now as a programmer in some company. Though they were programming books but they were not written to show you the reality of what happens when you use scp to copy file over a network. How that scp program looks like and Why it is so strange at 1st. Those books will never tell you ( in code ) that their could be errors, their could be failures, their could be some problem in making a connection, that, for obvious reasons, you have to use ” -ansi -pedantic -Wall -Wextra” while working with GCC but all we learned were TURBO C and a book on DOS 6.2 ( this shitty-book had cost me my precious 600 rupees )I put Knuth in the same category. He just explains, explains and explains the algorithms, with 100% Mathematics, in the most detailed manner ever done and leaves the applications of those algorithms up to you. They are not designed for people like me. [1]

If you don’t know then I want to tell you that Donald Knuth is a child-prodigy. When he was in 7th grade, he was solving mathematical problems of his seniors in higher-grades [2]. I am not a child-prodigy, I am an average man, with an average IQ. I could have have failed the Mathematics exam of 10th class in 1996. I was very poor at Mathematics and I was expecting only 22 or 34 marks out of 100 and anything less than 33 means you have failed the exam but I purchased 2 very heavy-books of Mathematics, which our teacher called best references books for 10th class and studied them at the cost of my health for 2 months and then I got 83 marks πŸ™‚ but those health problems, the pain in the neck are still with me after 12 years 😦 ) . Since I was weak in Mathematics may be thats why I don’t like Knuth’s style πŸ˜‰ . Knuth explains algorithms from one angle while not from the other. He explains everything, very diligently, in a detailed manner, from his angle, from his viewpoint (which is Mathematical and Mathematics is what he is good at) and he totally ignores the other viewpoints. On comp.programing, someone said this:

If you want to know how to take simple things and make them as as obtuse as humanly as possible, read Knuth.

user923005 at comp.programming said this to me when I said in a post that I don’t like academic texts:

A pity. Knuth is clearly the best computer science author ever, period, no question about it. But also quite academic in his approach.

On the other hand:
American Heritage Dictionary : academic (ak’?-dem’ik) adj.

1. Of, relating to, or characteristic of a school, especially one of higher learning.

2.
A. Relating to studies that are liberal or classical rather than technical or vocational.
B. Relating to scholarly performance: a student’s academic average.

3. Of or belonging to a scholarly organization.

4. Scholarly to the point of being unaware of the outside world. See Synonyms at pedantic.

5. Based on formal education.

6. Formalistic or conventional.

7. Theoretical or speculative without a practical purpose or intention. See Synonyms at theoretical.

8. Having no practical purpose or use.

If you are speaking of the 4th, 7th or 8th form, then I quite agree with you.

Knuth uses Mathematics to present his ideas while those very ideas can be presented without much Mathematics. As per my experience, Programming is a totally different phenomenon, its about analyzing problems, finding solutions and recognizing the patterns before you even start to code, as a daily-coder, recognizing the design of the solution and structure of the whole basic idea that will write in your favorite programming language. And this explanation has nothing to do with Mathematics. Hence one can explain the algorithms in as detailed manner as of Knuth from another viewpoint, like recognizing to what pattern problem belongs. Do not confuse the word Pattern with Design-Patterns of OOD, patterns mean simple way to generalize a problem and finding the category to where it belongs or create a new category of patters by using your experience with analyzing problems and then you can use whatever interface you have to show that design/solution/connection on paper. That could be a diagram, or a formula represented using ( x, y or + etc., which is Mathematics) or you could use something else I am not aware of. Then you will be a creator πŸ™‚

I think understanding K&R2 is as much hard as understanding Knuth ( I did not say reading, I said understanding) , thats what I have felt and I have enjoyed the painful and difficult K&R2 while I contribute all of my frustrated moments to Knuth. Don’t ever think that K&R2 is easy, it is a small book, at a very cheap price and it will extract the brains out of you. You are warned.


> Mathematical analysis of algorithms is of
> fundamental importance; however, it is not
> for everyone. The fact that you can compile
> your source codes very fast, the fact that
> you can search a string within a text very
> fast, it all comes down to the mathematical
> analysis of algorithms.

What is the point of using Mathematics to represent algorithms ?

Mathematics is a traditional and quite good tool to represent some underlying concepts and fundamentals in formal way. Maths is a formal way of representing something more abstract than Maths:

  Underlying Fundamental Concepts & Ideas
                |
                |
                |
  Formal Representation of those ideas using
  Maths.
                |
                |
                |
    Algorithms  & Data Structures

what I am saying is the understanding underlying fundamental ideas which are more abstract and basic than Maths is more important than Maths. Mathematical is used as a tool in computer programming to represent some ideas and I think anyone will be in better position if he tries to understand those ideas first but people put whole of their emphasis on Maths which is not the way to do programming. Take one more example:

        
       N - 1
    ------------
     \
      \
       \
        /
       /
      /
     -------------
        i = 0

that is sigma, a mathematical notation to represent that “i” goes from zero to N. Now look at this:

for( int i = 0; i != N; ++i );

this does the same thing, a for loop. That sigma, the Mathematical notation, is a for loop in disguise but nearly all of the books that teach algorithms use sigma instead of for loop. The Sigma belongs to Mathematics and for loop belongs to programming but the main underlying idea is to start i at zero and then increment it till it is less than N. Sigma is used to formally represent that concept. you can’t use sigma inside C++ code, you can use only for loop πŸ˜‰

> Sometimes, we take for granted what went
> into creating the high-level stuff that
> we enjoy so much. Find it out yourself:
> imagine a compiler without automata theory.

again, automata theory is modal to represent more abstract and more important concepts.

> Mathematical analysis of algorithms is of
> fundamental importance; however, it is not
> for everyone.

yes, it is not for people like me, who like to invest their time into practical ways of understanding programming and much more interested in representation of those ideas into code, rather than pondering on learning formal ways to express those ideas.

But I will still say, as of 2008, I think Maths is the best tool to represent ideas formally, I don’t think any other tool will fit better. First one needs to understand those underlying ideas and concepts that Mathematics represents and then one needs to understands Maths, if he is interested in representing those ideas formally, I don’t. I represent them directly using code. May be, in future, I create some tool to represent basic ideas informally πŸ™‚

 

 

[1] Don’t think that Knuth is only theoretical, some time ago, he wrote a very specialized algorithm for a quite powerful chess program)

[2] Discrete Mathematics and Its Applications by Kenneth H. Rosen, The author gives brief histories of most mathematicians in his book.

Copyright Β© 2008, Arnuld Uttre, #331/type-2/sector-1, Naya Nangal, Distt. – Ropar, Punjab (INDIA) – 140126

Verbatim copying and distribution of this entire article are permitted worldwide, without royalty, in any medium, provided this notice, and the copyright notice, are preserved.

Advertisements

18 Comments »

RSS feed for comments on this post. TrackBack URI

  1. Yes, thank you… too many times things are explained in a mathematical style, I’d really like to look at the implementation to understand what’s really going on.. there is definitely a language barrier between mathematics and programming.

  2. What you want are books that help you code. You are looking for a solution to your immediate problem, not knowledge that will help you develop those solutions yourself.

    Knuth’s books are dense and difficult. They are not about programming, in the sense you define it. They are about the science of software development. Computer Science is at it’s core a mathematical discipline. Programming is a craft. There is nothing wrong with pursuing either, or both. And, a solid foundation in computer science will benefit most programmers. But, it is possible to become a perfectly good programmer without becoming a computer scientist.

  3. are you _actually_ into lisp? your dislike for theoretical stuff suggests otherwise.

  4. Good article. What are the names of the two math reference books your teacher recommended? Thanks.

  5. nice post …

  6. I agree with your dislike for presenting algorithms through maths. I’ve programmed since I was 5 (which means I’ve got 28 years of programming behind me – I’m feeling old), and developed software professionally for a long time. I wrote my first compiler at 13. I program for fun as well as professionally. I enjoy reading research papers to relax and learn new stuff…

    All in all, I’m happy to go into some very complicated theoretical aspects, and have the experience to back it up.

    But few things annoy me more than computer science papers that are littered with mathematical annotation when they don’t need to be. Which is to say almost never.

    It particularly pisses me off because most of it is full of unstated assumptions which you can easily enough guess at, but which means that their “maths” tend to be no more a rigorous definition than what they could have achieved as easily in English. If the goal was to present, say, a logically sound proof for something, then yes, mathematical annotation can sometimes be defended in CS papers.

    But that’s rarely how it’s used. It’s used out of what seems like laziness, as shorthand, to avoid explaining something. Often that “shorthand” actually obscure a lot of missing details that would have been obvious had the ideas been written out in English.

    When I was writing my MSc. thesis I read far more papers than usual, and I noticed that in the field I was looking at (OCR) there was absolutely no correlation between the use of maths in the papers and the complexity of the subject matter or the quality of the research.

    Some fairly trivial papers were full of maths for no good reason, and many brilliant papers, including papers employing statistical methods, required no maths beyond that which an average 12 year old would understand.

    Of course a paper with lots of Greek symbols look a lot more impressive (that’s sarcasm, for the impaired…).

    I’ve come to see maths in CS papers as either a crutch used by insecure people or evidence of lack of communications skills (the latter from people who are often brilliant, but don’t realize that they are cutting of a huge chunk of a potential audience that would have no problems understanding their papers if they were less maths heavy).

    I do see people that push the view that we should just “suck it up” and learn the maths, but I’d rather spend that time on computer science or development. I do occasionally pick up some new maths to understand a particularly compelling paper, but it’s rare to find something that’s so groundbreaking it’s worth spending a lot of time learning maths concepts that are otherwise useless for me just to read some paper.

  7. There’s a difference a computer scientist and a software engineer, and obviously Knuth is a computer scientist.

    And that’s it. Nothing really bad about it, just putting it in the proper context.

  8. > There’s a difference a computer scientist and
    > a software engineer, and obviously Knuth is
    > a computer scientist.

    and I thought there is a difference between Computer Scientist, Programmer and Software Engineer, and obviously Knuth is the Computer Scientist.

    I have also seen claims that Programming and Software Development are different things but I don’t think they differ as much as the 3 things I have mentioned.

  9. … Replying to “Vidar Hokstad”

    thanks for your precious time in putting such a lengthy and meaningful comment.

  10. Mathematical analysis of algorithms is of fundamental importance; however, it is not for everyone. The fact that you can compile your source codes very fast, the fact that you can search a string within a text very fast, it all comes down to the mathematical analysis of algorithms. Sometimes, we take for granted what went into creating the high-level stuff that we enjoy so much. Find it out yourself: imagine a compiler without automata theory. This does not mean that everyone should learn mathematical analysis of algorithms to do their job. People in the IT industry should stick to books which are designed to meet their current requirements and trends.

    Knuth does not write books depending on what the market demands; he is writing about things that he believes to be of fundamental importance; things that will survive the test of time, when C#, Java, C++ or Perl is considered outdated, or even impractical. This is precisely the reason why he chose (see, Preface of “TAOCP: Fundamental Algorithms”) to use a virtual machine (MIX and MMIX) instead of a real machine: virtual machines are conceptual, hence, it only requires imagination or simulation to recreate the idea; real machines, on the other hand, are significant only for that time (imagine a student who wants to try out a program written for the ENIAC machine).

    To sum up, if you are using algorithms to satisfy an IT demand, stick to the books which were designed for IT people. There are plenty available (every bookstore has an IT section). If you want to know why certain things are done is a particular way, read books that contain thorough mathematical analysis of the algorithms. Knuth’s books are brilliant in the latter!

  11. > Knuth does not write books depending on
    > what the market demands;

    well, I never said that he writes books on what market demands. What market demands is always some stupid things like Windows XP. Knuth is much more fundamental than them πŸ™‚

    I have edited the original post to include an answer to your comment.

  12. It’s music, whether you can read tabs, scores, or play your instrument without any sheets in front of you.

    Designing algorithms, analyzing algorithms, programming – is applied mathematics. Not everyone involved needs to be fluent with rigorous mathematical notations, most of the time having a good intuition is enough.

    However, you probably would not program computers today, if it were not for brilliant mathematicians like Alan Turing. Also, if computer scientists and mathematicians did not invent all the high level tools you as a programmer can exploit and plug together, there would not be many IT jobs either πŸ™‚

    If you are not interested in doing average case, or not even interested in doing worst case analysis of your programming constructs, than you do not need to study Knuth. That’s fine. Most of time, programmers plug together algorithms and protocols that are already analyzed and optimized for them. However, if you do invest time to study the techniques introduced by Knuth, you might become a better programmer. You might also get frustrated and give up altogether, depending on your personality.

  13. Just do it, it’s not that hard, you can handle it, and it will make you a better person.

  14. Maths is totally important for programming. You may not like math, but in order to solve problems you totally need math & you will/have to use it one way or other. Directly or indirectly.

    Let me give you a simple example. Write a simple program which calculates sum of all numbers upto 7 trillion, which are divisible by 7.

    Use formal method & for loops, you will probably get the answer in a weeks – months [depending on CPU you use].

    Use mathematical induction & series summation, you will have the answer in hours πŸ˜‰

    But I like the way you have written the article & agreeing being biased about his books. To each his own. Btw I have read & totally like K&R2 as well Knuth’s books.

    cheers !

  15. You mentioned someone who sent you an email saying Knuth was the best computer scientist ever, period, etc. I would never take anyone seriously who made that type of statement. It shows a lack of judgement and frankly a lack of intelligence. (Unless it is someone related to Knuth or in love with Knuth, in which case it is bias).

    I don’t care for Knuth’s snide writing style; his attempts to be humurous fall flat. I much prefer Wirth. Wirth covers all the basics of programming, from beginning concepts to writing a compiler, in one reasonably sized book called Algorithms Plus Data Structures Equals Programs. Wirth has very clear ideas and presents them in a clear and disciplined manner. He doesn’t go out of his way to be a “personality” as some writers do.

  16. Can you recommend a succinct resource that explains Math Notation into Code (i.e more examples of ‘Sigma’ => for( int i = 0; i != N; ++i ); or perhaps write a “How to code Math Symbols” primer. I came across you post looking for a “cheatsheet” not much luck yet.

    • I really do not know. That summation sign and for-loop connection I learned on comp.lang.c while trying to learn C language. You can try books on Mathematical Modelling and google for Mathematical Modelling and Computation. I am sorry but that is all I can help. Thanks

  17. I think Knuth’s books are not about “programming”, but about “analizing algorithms”: efficiency. They’re not about design, about best practices, or about reusable software or something like that. It’s about analizing algorithms, and that can’t be done without maths, which brings you to the point of representing algorithms you want to analyse in a formal way.

    If you are writting critical code, perhaps you do must read Knuth. If you want to be a better programmer in general, so don’t.

    Comparing a book about algorithm analysis and programming is like comparing books of cellular biology with healthy cooking.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.

%d bloggers like this: