MySQL Stored Procedure Programming 206
Michael J. Ross writes "MySQL may be the most popular open source relational database management system (RDBMS) in the world, but during the first decade of its existence, it lacked support for stored programs, i.e., store procedures, functions, and triggers. The major commercial RDBMS vendors — including Oracle, IBM, and Microsoft — could point to this deficiency as reason enough to choose their proprietary systems over MySQL or any other open source system, such as PostgreSQL. But with the release of MySQL version 5.0, in October 2005, the "little database engine that could" dramatically improved its position against the competition. The most comprehensive discussion of these new capabilities is in the book MySQL Stored Procedure Programming." Read below for the rest of Michael's review
| MySQL Stored Procedure Programming | |
| author | Guy Harrison and Steven Feuerstein |
| pages | 636 |
| publisher | O'Reilly Media |
| rating | 9 |
| reviewer | Michael J. Ross |
| ISBN | 0596100892 |
| summary | A comprehensive guide to developing MySQL stored procedures, functions, and triggers. |
Written by Guy Harrison and Steven Feuerstein, and published by O'Reilly Media in March 2006 under the ISBNs 0596100892 and 978-0596100896, this book is the first one to offer database programmers a full discussion of the syntax, usage, and optimization of MySQL stored procedures, stored functions, and triggers — which the authors wisely refer to collectively as "stored programs," to simplify the manuscript. Even a year after the introduction of these new capabilities in MySQL, they have received remarkably little coverage by book publishers. Admittedly, there are three such chapters in MySQL Administrator's Guide and Language Reference (2nd Edition), written by some of the developers of MySQL, and published by MySQL Press. Yet this latter book — even though published a month after O'Reilly's — devotes fewer than 50 pages to stored programs, and the material is not in the printed book itself, but in the "MySQL Language Reference" part, on the accompanying CD. That material, in conjunction with the online reference documentation, may be sufficient for the more simple stored program development needs. But for any MySQL developer who wishes to understand in-depth how to make the most of this new functionality in version 5.0, they will likely need a much more substantial treatment — and that's exactly what Harrison and Feuerstein have created.
The authors are generous in both the technical information and development advice that they offer. The book's material spans 636 pages, organized into 23 chapters, grouped into four parts, followed by an index. The first part, "Stored Programming Fundamentals," provides an introduction and then a tutorial, both taking a broad view of MySQL stored programs. The remaining four chapters cover language fundamentals; blocks, conditional statements, and iterative programming; SQL; and error handling. The book's second part, "Stored Program Construction," may be considered the heart of the book, because its five chapters present the details of creating stored programs in general, using transaction management, using MySQL's built-in functions, and creating one's own stored functions, as well as triggers. The third part, "Using MySQL Stored Programs and Applications," explains some of the advantages and disadvantages of stored programs, and then illustrates how to call those stored programs from source code written in any one of five different programming languages: PHP, Java, Perl, Python, and Microsoft.NET. In the fourth and final part, "Optimizing Stored Programs," the authors focus on the security and tuning of stored programs, tuning SQL, optimizing the code, and optimizing the development process itself.
This is a substantial book, encompassing a great deal of technical as well as advisory information. Consequently, no review such as this can hope to describe or critically comment upon every section of every chapter of every part. Yet the overall quality and utility of the manuscript can be discerned simply by choosing just one of the aforesaid Web programming languages, and writing some code in that language to call some MySQL stored procedures and functions, to get results from a test database — and developing all of this code while relying solely upon the book under review. Creating some simple stored procedures, and calling them from some PHP and Perl scripts, demonstrated to me that MySQL Stored Procedure Programming contains more than enough coverage of the topics to be an invaluable guide in developing the most common functionality that a programmer would need to implement.
The book appears to have very few aspects or specific sections in need of improvement. The discussion of variable scoping, in Chapter 4, is too cursory (no database pun intended). In terms of the book's sample code, I found countless cases of inconsistency of formatting — specifically, operators such as "||" and "=" being jammed up against their adjacent elements, without any whitespace to improve readability. These minor flaws could be easily remedied in the next edition. Some programming books make similar mistakes, but throughout their text, which is even worse. Fortunately, most of the code in this book is neatly formatted, and the variable and program names are generally descriptive enough.
Some of the book's material could have been left out without great loss — thereby reducing the book's size, weight, and presumably price. The two chapters on basic and advanced SQL tuning contain techniques and recommendations covered with equal skill in other MySQL books, and were not needed in this one. On the other hand, sloppy developers who churn out lamentable code might argue that the last chapter, which focuses on best programming practices, could also be excised; but those are the very individuals who need those recommendations the most.
Fortunately, the few weaknesses in the book are completely overwhelmed by its positive qualities, of which there are many. The coverage of the topics is quite extensive, but without the repetition often seen in many other technical books of this size. The explanations are written with clarity, and provide enough detail for any experienced database programmer to understand the general concepts, as well as the specific details. The sample code effectively illustrates the ideas presented in the narration. The font, layout, organization, and fold-flat binding of this book, all make it a joy to read — as is characteristic of many of O'Reilly's titles.
Moreover, any programming book that manages to lighten the load of the reader by offering a touch of humor here and there, cannot be all bad. Steven Feuerstein is the author of several well-regarded books on Oracle, and it was nice to see him poke some fun at the database heavyweight, in his choice of sample code to demonstrate the my_replace() function: my_replace( 'We love the Oracle server', 'Oracle', 'MySQL').
The prospective reader who would like to learn more about this book, can consult its Web page on O'Reilly's site. There they will find both short and full descriptions, confirmed and unconfirmed errata, a link for writing a reader review, an online table of contents and index, and a sample chapter (number 6, "Error Handling"), in PDF format. In addition, the visitor can download all of the sample code in the book (562 files) and the sample database, as a mysqldump file.
Overall, MySQL Stored Procedure Programming is adeptly written, neatly organized, and exhaustive in its coverage of the topics. It is and likely will remain the premier printed resource for Web and database developers who want to learn how to create and optimize stored procedures, functions, and triggers within MySQL.
Michael J. Ross is a Web programmer, freelance writer, and the editor of PristinePlanet.com's free newsletter. He can be reached at www.ross.ws, hosted by SiteGround.
You can purchase MySQL Stored Procedure Programming from amazon.com. Slashdot welcomes readers' book reviews -- to see your own review here, read the book review guidelines, then visit the submission page.
MySQL aren't trustworthy (Score:3, Insightful)
It's lack of trust.
MySQL started off making their fast little datastore that uses a little SQL syntax, and they told everyone under the sun that if MySQL didn't do it, you didn't need it, and ought to design around it.
Referential integrity, ACID compliance, enforcement of rules... every step along the way, they tell you you don't need it and you ought to push the missing logic to the next tier, until they get it, if they get it, then they're so great.
Couple that with the Project Mayo/DivX corporate structure, then place Postgresql next to the whole shebang as a superior alternative, and the arguments for giving MySQL any of your attention become increasingly small.
MySQL owes its success to the fact that it was always so simple to choke it off when you were giving it to people on $5/month hosting plans, so it became popular among the very cheap.
Period.
The parent comment is a classic example of FUD (Score:2, Insightful)
Re:MySQL aren't trustworthy (Score:3, Insightful)
I call shenanigans. Only recently has it become superior. Documentation isn't just a feature, it's a necessity, even with the open source world, and Postgresql's documentation is finally getting to a good point. I tried numerous times to install and use this "superior alternative" with bad documentation(it was all there, but in backwards, confusing steps, how to access your database was before creating it, etc..) it was just disorganized. Privileges were a bitch with very confusing instructions. Yeah, if you knew it, superior alternative, if you didn't, RTFM, oh wait, that didn't actually help. Now the MySQL can compete better on features(not completely mind you, PostgreSQL still has some important features missing from MySQL) the documentation is starting to improve a lot on the postgresql front.
Deciding if MySQL is an option (Score:4, Insightful)
Re:MySQL vs Firebird (Score:5, Insightful)
My understanding is that Firebird is by any measure more sophisticated than MySQL, but it lacks the "critical mass" of users that makes it attractive to people who need to be sure they can get ready support for their software. MySQL is available on just about every cheap hosting provider around, too, which means that a lot of ready-built open source Web apps target MySQL as their database of choice.
Put it to you this way: If the features of MySQL are "good enough" for the application you want -- which arguably goes for most of the Web apps out there -- why would you not choose MySQL? There are times when swimming upstream is a noble effort, but generally all it gets you is tired.
If, on the other hand, you have a specific application for which you need a relational data store and you need higher-end RDBMS features, by all means, choose Firebird. Only I think in those cases most people still choose PostgreSQL, for more or less the same reasons as mentioned above.
Re:Stored procedures BAD... story (Score:5, Insightful)
Rest assured that if you were one of my developers you'd be out the door in no time. The application you're describing here cannot be anything north of trivial if you were able to just switch a connection string and all your messy inline SQL statements continued to run without any changes whatsoever (which I seriously doubt). Not everyone writes trivial applications.
There are reasons other than "fast" to using stored procedures. I've seen enough misuse of SPs, functions and triggers to fill up a book, but when done correctly they are simply superior to the alternative in just about every way.
Re:Stored procedures BAD... story (Score:5, Insightful)
I'm glad I don't work for you, then. Stored procedures are a tool, like any other, that come with their own set of pros and cons. In some situations, the ability to quickly migrate to an alternate database outweighs the benefits that stored procedures may provide. In other situations, it doesn't. The decision to use them or not should be based solely on business requirements, rather than the irrational hatred you seem to be using as a guide.
Re:Stored procedures BAD... story (Score:3, Insightful)
And you'd get sued shortly thereafter for unfair dismissal
There are plenty of good reasons to use stored procedures in a database - they simply wouldn't exist in every serious database if they were not useful. Speed isn't the only reason to use a stored procedure, they can also make some inter-application integration much easier and allow someone in control of the data model to abstract the data collection routines from the underlying data structure - which can be very, very useful. I don't want some Java programmer thinking he knows the best way to get at data in Oracle - using a stored procedure to pass complex data structures back allows tweaking of the select statements, indexes etc... without any testing required of the app layer.
D.
Stored procedures and data integrity (Score:3, Insightful)
As a database engineer, I would *definitely* fire anyone who didn't use these tools to maintain data integrity.
Re:Stored procedures BAD... story (Score:5, Insightful)
Re:Stored procedures BAD... story (Score:4, Insightful)
So answer me this: Now that you've made the investment to move to Oracle, can you reasonably foresee moving back to SQL Server? Seems unlikely to me. So if you'd chosen the right tool for the job in the first place, the migration wouldn't have been a problem and you would have been free to use other right tools where appropriate, e.g. stored procedures -- right?
Really, stored procedures have their place. Oracle tends to over-sell them, but to ignore them completely seems like one more step backward in the last n years of best practices and lessons learned.
It isn't about speed (Score:5, Insightful)
Middle-tier application layers are great to help pull data together, present model-specific views of data (say, OO/R mapping, if that is what gives you a chubby), and provide update interfaces. They can even do a first pass at making sure the database will accept the data.
But.
The data logic should be stored firmly in the database logic itself, using stored procedures and triggers and rules. RDBMS engines are *designed* to keep your data in good shape. Use them in that fashion, and you will be plenty happy. Don't, and you risk losing data integrity, and you sacrifice security.
Names matter sometimes (Score:5, Insightful)
Re:What's your opinion (Score:5, Insightful)
On the one hand, folks who are trained to think in terms of data relationships and database development are liable to want to put as much as possible right in the database.
On the other hand... well, forget it. There is no other hand, it's more like a shifting mass of appendages, like Shiva [wikipedia.org] on an acid binge. The other extreme are people like David Heinemeier Hansson, who want to use database to store information and otherwise stay the hell out of the way - as I believe he put it once, to be as dumb as possible.
That may be a minority view, but the minority isn't exactly vanishingly small. eBay, for example, not only skips out on transactions (you know, the thing people like to beat up on MySQL for not having years ago), but doesn't even enforce referential integrity in the database. Stored procedures are right out. (warning: reference is a PDF [singleobject.org].)
My own personal, under-informed and probably worthless view: I like enforce referential integrity where I can in the DB, and transactions are a good thing, but stored procedures are better left to maintenance and administration functions than anything involving business logic.
Re:Stored procedures BAD... story (Score:2, Insightful)
Re:What's your opinion (Score:1, Insightful)
My approach is ususally to get a somewhat abstracted view of the data out of a stored procedure and then if there is heavyweight processesing of data required send it to an application for processing.
Putting logic in stored procedures makes it easy to fix problems and add new features or customizations without having to recompile source code in the field. However that same logic unless you take steps to encrypt stored procedures (This is supported in Oracle and some other systems) also gives away parts of your application, in some instances very vital parts.
For simple pulls of data I might use stored procedures for abstraction purposes or if its application specific a simple select statement generally works the best across different platforms provided you need to support several RDBMS systems.
Sometimes I don't use them for technical reasons such as when issuing updates that need to have their row counts checked for consistancy reasons. I ususally don't put that in stored procedures because of inconsistancies with various RDBMs and database drivers.
We have procedure converters which automatically convert the syntax of a common procedure syntax to various RDBMS platforms. Its quite a complex proposition in some cases but investments like these can pay off for large projects where procedure use can actually be a benefit for supporting multiple RDBMS targets.
On to my take on MySQL... I flat out refuse to support this platform. Its less than a joke, its **DANGEROUS**. Who needs DRI, sane quoting rules that screams injection attack or any reasonable notion of field validation. I'm sure like the text ODBC driver its great for a multitude of purposes. Just not mine..not ever in the forseeable future. I honestly do hope they continue to improve their product. One day I have no doubt it will be less of a joke.
If my MySQL is really the most popular open source database platform then IMHO its a sorry reflection on the community.
Re:Deciding if MySQL is an option (Score:5, Insightful)
If you want a DATABASE it's not even on the table, no pun intended. The whole point of a RDBMS is that it isn't a simple store, it's a mechanism for ensuring that your collection of data is always in a sane state even if you've brought in some clueless interns for the summer or you have a disgruntled employee. Or even if you just have two developers (or one developer with a poor memory).
ACID and referential integrity are two items that are absolute requirements. Calling something without these features a 'relational database' puts you in the same pointy-hair territory as Dilbert's boss saying that he heard the 'mauve' databases were best.
Attribute constraints, triggers and stored procedures aren't as necessary, but they're still extremely powerful ways of ensuring the sanity of your data. Use a trigger to update a 'updated_on' field, don't just trust the developer to always update it. Use an attribute constraint to ensure that you color field is always 'R', 'G' or 'B' (or just use referential integrity to point to a color table).
Finally, to address a question asked elsewhere stored procedures are extremely powerful security tools. By now everyone should know that using string concatenation to prepare queries is a Very Bad Idea. A potentially "don't let the door hit you in the butt on your way out" Bad Idea. Prepared statements are better, but how can you enforce it?
Stored procedures give you an alternative. Drop INSERT and UPDATE rights and force everything to go through a corresponding stored procedure. It's a little more work but it should eliminate any risk of SQL exploits. (It's not a 100% guarantee since you can't eliminate the risk that the database itself can be compromised by carefully selected parameters.)
Re:What's your opinion (Score:3, Insightful)
I am fully aware that I would be flamed to death by ACID zealots, but I will again repeat nothing.
We are in 2007. Using non-OO fortran-like obscenities dated 1993 instead of looking at the calendar and getting a grip with reality gives you exactly that - nothing.
There is a reason why banks, govt, etc has been looking more and more at middleware layers on top of databases. You get the same you get with stored procedures while still having modern data and code representation.
Similarly, the little engine that could should have actually looked harder at making things that matter like isolation levels, foreign keys, etc work with all of their engines and not just one. Which they lost to Oracle anyway and which had no means of online backup (within the same license). They did one right thing at one point by introducing stored Java, C++ and Python procedures. It is a pity they did not expand on that and degraded into the fortran age instead.
Re:MySQL vs Firebird (Score:4, Insightful)
Easy. The nimrod who wrote the application that you want/need to run didn't make the app database agnostic, so you are stuck with MySQL because "it's more popular". It's a catch 22. Until more folks start writing database agnostic apps, lots of us will get stuck using MySQL in places where we might prefer other databases. And so that will perpetuate MySQL being seen as being more popular... Argh.
Re:What's your opinion (Score:3, Insightful)
Rule #1 of Software Development in the 21st century:
Don't put any business logic into sprocs.
Why?
It's very simple really, and very logical.
1) Only CRUD operations should be in sprocs. These sprocs are designed to allow someone to create/read/update/delete data in the tables in the database without giving the user access to or knowledge of the underlying table structure. Use application roles (approles) to allow access to the tables. This keeps people from loading up your tables with Excel and walking all over your data. Also, this prevents SQL injection attacks, as your sprocs will choke all over them and nothing will get trampled. Any time that you put SQL statements in a form, you are opening yourself up to SQL injection attacks, unless you have specifically guarding against them, which requires even MORE work. Not giving your users knowledge of the underlying table structure makes it easy for you to change that structure, should you (or the DBA) need to.
2) The second that you place a business rule into a sproc, the users will want the rule changed. This means that you now have to go back to the sproc to update it. Why is this bad? Because sprocs are almost always under the control of the DBA. This sometimes isn't that bad, but most of the time there is a layer of politics involved between DBAs and Software Developers. What usually happens is that the dev will make the change to the business logic in the middle (or even worse, GUI) tier, thereby doing twice as much work. This happens way more often than you think it does.
3) Adding Database servers to a cluster isn't cheap. If you have a lot of users, and you have had to cluster your servers to accommodate more users, you'll have to add more database servers as you add more users. This is expensive, especially if you are using SQL Server or Oracle. Adding more middle-tier servers is much cheaper, being that you only need the underlying OS. Not to mention that there is a diminishing point of returns when adding database servers.
I hope that this makes sense to you, and helps you build arguments in the future for NOT allowing devs to put business logic into the sprocs!
-Steven
Re:MySQL aren't trustworthy (Score:4, Insightful)
Only recently has it become superior. Documentation isn't just a feature, it's a necessity, even with the open source world, and Postgresql's documentation is finally getting to a good point.
Unless something dramatic has happened to MySQL documentation since I last used it (version 4.1), then PostgreSQL has the better documenation and has done since as long as I've been using it. The PostgreSQL documentation reads like a coherent set of books, while MySQL documentation seems to be a random mess that grew out of some simple README file. As for the software itself, PostgreSQL has always been superior - a simple comparison of features alone bears this out, but if you want to go further take a look at the code as the same distinction is found there.
Re:Deciding if MySQL is an option (Score:3, Insightful)
Feature comparison checklists are so 1985ish (Score:3, Insightful)
Why does everyone love CSS for separating content from format
Presentation - Logic - Storage. Clean and simple. Stored procedures and triggers are nothing more than DB-CRACK. Easy to use in a "pinch" yet keeps you addicted forever.
The concept of moving trigger/stored procedure coding out of the database engine is the scariest thought for Oracle/MS/IBM....yet it works surprisingly well in the freebie MySQL.
This subject is so deep and broad it leaves the mega dollar database companies room for more FUD the HS in NYC in 1905 (check that one out in your database).
SCALE THAT IN YOUR BOARDROOM!
Re:What's your opinion (Score:2, Insightful)
I am fully aware that I would be flamed to death by ACID zealots, but I will again repeat nothing.
You mean, people who actually know something about data modeling?
We are in 2007. Using non-OO fortran-like obscenities dated 1993 instead of looking at the calendar and getting a grip with reality gives you exactly that - nothing.
Well, sonny, this might be before your time, but do you know how they wrote apps in the 60's and 70's? That's right. Everything was in the apps. Everybody was creating their own little toy DBMS layer and calling it something different. Data integrity was nowhere to be found. Application independence was a joke. There was no relational model to tie together and generalize the network and hierarchical models.
Now in 2007, we see developers doing things the same way. Of course, instead of "network DBs" we have "object DBs", and instead of "hierarchical" we have "XML". The previously unnamed pseudo-DBMS layer is now called "ORM layer" or "persistence layer" or "middleware". Instead of general queries, we get XML-RPC interfaces and navigational, element-at-a-time retrieval. People think that pulling out objects in a "for loop" is somehow superior to asking the DBMS for what you need in a single query, letting it optimize the query, and retrieving it all at once.
As far as geniuses like you are concerned, the last 3-4 decades of data management research just didn't happen. Except maybe the functions are all lowercase now, so you can make fun of old-fashioned all-uppercase syntax.
Congrats. You've come a long way, baby.
Re:The primary purpose of stored procedures (Score:3, Insightful)
Stored procedures are extremely useful for security, to reduce network traffic to the client or application server, to ensure consistency, to take advantage of database-specific features, and most importantly to ensure data integrity.
Re:What's your opinion (Score:4, Insightful)
Do put critical enforceable bussiness logic (and I mean *bussiness* logic, not application logic) as near as possible to the data (and that usually means SP).
"The second that you place a business rule into a sproc, the users will want the rule changed"
That's true. And that's an advantage of stored procedures: they won't be able to change them (that's why I talked about Critical-Enforceable-Bussiness-Logic).
"Why is this bad? Because sprocs are almost always under the control of the DBA"
Why is this good? Because sprocs are almost always under the control of the most data management knowledgeable techie round there.
"most of the time there is a layer of politics involved between DBAs and Software Developers"
And that's good because that refrain them from their need (either because management pressure or pure ignorance) to do the thing the Real Stupid Way just because it's the fastest/the only way they know.
"What usually happens is that the dev will make the change to the business logic in the middle (or even worse, GUI) tier, thereby doing twice as much work"
So you say that such a bussiness logic should go into the middle tier instead of the RBDM because in that case the developer would do a stupid thing -like expressing it in the middle tier? This is both a circular argument and one more reason to put those kind of rules *out* of the reach of such a stupid developer.
"Adding Database servers to a cluster isn't cheap. If you have a lot of users, and you have had to cluster your servers to accommodate more users..."
Then you need to add more database horsepower. If you need to push or pull more data from the place the data is stored... well, you need it. No middleware is going to change that.
"Adding more middle-tier servers is much cheaper"
Yes. And buying a 1GB pen-drive is even cheaper than that, having in common with your "solution" that they both have nothing to do with the problem at hand. You either need to pull/push more data from where it resides or you don't. In the first case you always need more database horsepower; in the second you won't need it (and if cacheing is a good solution to your current problem then no, your problem is not that you need to push/pull more than provided and no, a cacheing solution is not a "middleware").
Resuming:
You will want stored procedures for your critical enforceable bussiness logic because:
*Developers tend to suck regarding data access/management (design, integrity or efficency); DBAs a little less so (it's the work they are focused on, after all).
*No matter how good your bussiness logic implementation on your new shiny app is, tomorrow it will be developed a new app that will access the same dataset (like the example you talk about people pumping data from Excel -presumably instead of through your pretty web app that tried to enforce this or that rule) and will surely break some bussiness logic or at the very least you will need to reimplement them again. If you don't want this to happen, enforce the rules as near to the data as possible so there's no way to overrun them.
*Your managers are crazy. It's much more probable that your new manager (or the old one if it happens to read the last shiny brochure from "PHB's IT") decides to rewrite all your
*Anyway, your mileage is your mileage: think it hard, think it deep and reach your own conclusions.