The tragedy of SQL

They are making a decent decision. SQL is a *fucking terrible* language, which I don’t blame them for not wanting to learn.

The whole industry, programming languages, infrastructure, everything would have developed differently if relations were a natural, pleasurable thing to use in any programming language. Like an Array, or a Hash.

On 9/13/21 11:51 PM, Guyren Howe wrote:

They are making a decent decision. SQL is a *fucking terrible*
language, which I don’t blame them for not wanting to learn.

The whole industry, programming languages, infrastructure, everything
would have developed differently if relations were a natural,
pleasurable thing to use in any programming language. Like an Array,
or a Hash.
On Sep 13, 2021, 22:45 -0700, Hemil Ruparel
<hemilruparel2002@gmail.com>, wrote:

SQL is not the problem. Problem are the devs. I love SQL. I hate
orms. The problem with databases is people refuse to treat it as the
entity it is and want to use their beautiful OO system. Problem is
databases are not OO. We need to recognize that and treat databases
as databases.

All languages are fucking terrible.  There are thousands of the them
because some people bump into a feature they don't like and run off an
make another fucking terrible language.  For the love of God, please
don't be one of those people.  The rest of us find languages we can
abide and do productive things with using features we like and avoiding
those we don't.  I've always felt it was no small miracle the vendors
managed to agree to ODBC/JDBC driver specs (even though the SQL language
definition is "more like guidelines").  Go scream at the DOM and JavaScript.

Many languages are awesome. I'm always astonished at what great
things people have come up with, over the years; it's been a
wonderfully fertile field. We would certainly not be better off if
we'd just buckled down, and used COBOL and FORTRAN... or even
relatively good languages like C, APL, and Lisp.

It is certainly possible to change too lightly, for small reasons.
That doesn't mean that forever enduring the same problems is a good
idea.

On Tue, Sep 14, 2021 at 2:18 AM Rob Sargent <robjsargent@gmail.com> wrote:

On 9/13/21 11:51 PM, Guyren Howe wrote:

They are making a decent decision. SQL is a *fucking terrible* language, which I don’t blame them for not wanting to learn.

The whole industry, programming languages, infrastructure, everything would have developed differently if relations were a natural, pleasurable thing to use in any programming language. Like an Array, or a Hash.
On Sep 13, 2021, 22:45 -0700, Hemil Ruparel <hemilruparel2002@gmail.com>, wrote:

SQL is not the problem. Problem are the devs. I love SQL. I hate orms. The problem with databases is people refuse to treat it as the entity it is and want to use their beautiful OO system. Problem is databases are not OO. We need to recognize that and treat databases as databases.

All languages are fucking terrible. There are thousands of the them because some people bump into a feature they don't like and run off an make another fucking terrible language. For the love of God, please don't be one of those people. The rest of us find languages we can abide and do productive things with using features we like and avoiding those we don't. I've always felt it was no small miracle the vendors managed to agree to ODBC/JDBC driver specs (even though the SQL language definition is "more like guidelines"). Go scream at the DOM and JavaScript.

--
Ray Brinzer

Is it possible that this is mainly an emotional discussion?

Raymond Brinzer schreef op di 14-09-2021 om 02:39 [-0400]:

On Mon, 13 Sep 2021, Guyren Howe wrote:

They are making a decent decision. SQL is a *fucking terrible* language,
which I don’t blame them for not wanting to learn.

SQL is not the problem. Problem are the devs. I love SQL. I hate orms.
The problem with databases is people refuse to treat it as the entity it
is and want to use their beautiful OO system. Problem is databases are
not OO. We need to recognize that and treat databases as databases.

Guyren/Hemil,

As a non-SQL expert who's used postgres since 1997 I've come to believe the
basic issue is that SQL is based on sets, neither procedural or object
oriented. Few people think in sets so they try to fit SQL into what they
know rather than understand the how sets work.

Rich

As a non-SQL expert who's used postgres since 1997 I've come to believe the
basic issue is that SQL is based on sets, neither procedural or object
oriented. Few people think in sets so they try to fit SQL into what they
know rather than understand the how sets work.

Yes, that's 100% correct. As per the general discussion, it's not like WE
decide what language/tech will be used, or not. If a sufficient number of
customers begin to offer well paid jobs for <whatever>, we will see a huge
run to learn <whatever>, and that's about it. In the end we all work for
the money, and language X may be the eighth wonder of the entire galaxy,
but if it doesn't deliver well paid jobs, nobody will bother learning it.

my 5p.

Berto

On Tue, Sep 14, 2021 at 12:32 AM Guyren Howe <guyren@gmail.com> wrote:

If I had $5 million to invest in a startup, I would hire as many of the core Postgres devs as I could to make a new database with all the sophistication of Postgres but based on Datalog (or something similar). (Or maybe add Datalog to Postgres). If that could get traction, it would lead in a decade to a revolution in productivity in our industry.

I've long thought that there is more algebraic type syntax sitting
underneath SQL yearning to get out. If you wanted to try something
like that today, a language pre-compiler or translator which converted
the code to SQL is likely the only realistic approach if you wanted to
get traction. History is not very kind to these approaches though and
SQL is evolving and has huge investments behind it...much more than 5
million bucks.

ORMs a function of poor development culture and vendor advocacy, not
the fault of SQL. If developers don't understand or are unwilling to
use joins in language A, they won't in language B either.

merlin

On Tuesday, 14 September 2021 14:06:13 BST Merlin Moncure wrote:

On Tue, Sep 14, 2021 at 12:32 AM Guyren Howe <guyren@gmail.com> wrote:

If I had $5 million to invest in a startup, I would hire as many of the
core Postgres devs as I could to make a new database with all the
sophistication of Postgres but based on Datalog (or something similar).
(Or maybe add Datalog to Postgres). If that could get traction, it would
lead in a decade to a revolution in productivity in our industry.

I've long thought that there is more algebraic type syntax sitting
underneath SQL yearning to get out. If you wanted to try something
like that today, a language pre-compiler or translator which converted
the code to SQL is likely the only realistic approach if you wanted to
get traction. History is not very kind to these approaches though and
SQL is evolving and has huge investments behind it...much more than 5
million bucks.

ORMs a function of poor development culture and vendor advocacy, not
the fault of SQL. If developers don't understand or are unwilling to
use joins in language A, they won't in language B either.

merlin

Back in the day, within IBM there were two separate relational databases. System-R
(which came from San Hose) and PRTV (the Peterlee Relational Test vehicle). As I
understand it SQL came from System-R and the optimizer (amongst other things) came
from PRTV.

PRTV ([1]https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV))" target="_blank" rel="noopener">https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)[1]https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)) did
not use SQL, and was never a released product, except with a graphical add-on which was
sold to two UK local authorities for urban planning.

So there are (and always have been) different ways to send requests to a relational DB, it is
just that SQL won the day.

--------
[1]: https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)

ORMs a function of poor development culture and vendor advocacy, not

the fault of SQL. If developers don't understand or are unwilling to

use joins in language A, they won't in language B either.

merlin

Back in the day, within IBM there were two separate relational
databases.� System-R (which came from San Hose) and PRTV (the Peterlee
Relational Test vehicle).� As I understand it SQL came from System-R
and the optimizer (amongst other things) came from PRTV.

PRTV
(https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)
<https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)&gt;)
did not use SQL, and was never a released product, except with a
graphical add-on which was sold to two UK local authorities for urban
planning.

So there are (and always have been) different ways to send requests to
a relational DB, it is just that SQL won the day.

Ah, lets not forget Mr Lane's favourite: quel

There was also QUEL. The original language for Ingress out of UCB.

On Tue, Sep 14, 2021 at 9:01 AM Rob Sargent <robjsargent@gmail.com> wrote:

ORMs a function of poor development culture and vendor advocacy, not
the fault of SQL. If developers don't understand or are unwilling to
use joins in language A, they won't in language B either.

Back in the day, within IBM there were two separate relational databases. System-R (which came from San Hose) and PRTV (the Peterlee Relational Test vehicle). As I understand it SQL came from System-R and the optimizer (amongst other things) came from PRTV.

PRTV (https://en.wikipedia.org/wiki/IBM_Peterlee_Relational_Test_Vehicle_(PRTV)) did not use SQL, and was never a released product, except with a graphical add-on which was sold to two UK local authorities for urban planning.

So there are (and always have been) different ways to send requests to a relational DB, it is just that SQL won the day.

Ah, lets not forget Mr Lane's favourite: quel

Sure, I quite like, er, liked quel also, being more mathematical and
formal. It's a shame it didn't make the cut. This is however a
telling example that standardization trumps purity once languages hit
a certain spot. There are many languages with dumb things that will
never get fixed :-). As they say, 'the devil you know'.

QUEL also uses idiomatic english for most operations, which I guess is
probably a contributing factor for developer resistance to SQL, since
native speakers are a minority of the earth's population. Oh well.

merlin

I started programming in 1967, and over the last 50+ years I've programmed
in more languages than I would want to list. I spent a decade writing in
FORTRAN on a GA 18/30 (essentially a clone of the IBM 1130) with limited
memory space, so you had to write EFFICIENT code, something that is a bit
of a lost art these days. I also spent a decade writing in COBOL.

I've not found many tasks that I couldn't find a way to write in whatever
language I had available to write it in. There may be bad (or at least
inefficient) languages, but there are lots of bad programmers.
--
Mike Nolan
htfoot@gmail.com

On 9/14/21 10:10 AM, Michael Nolan wrote:

I started programming in 1967, and over the last 50+ years I've
programmed in more languages than I would want to list.  I spent a
decade writing in FORTRAN on a GA 18/30 (essentially a clone of the
IBM 1130) with limited memory space, so you had to write EFFICIENT
code, something that is a bit of a lost art these days.  I also spent
a decade writing in COBOL.

I've not found many tasks that I couldn't find a way to write in
whatever language I had available to write it in. There may be bad (or
at least inefficient) languages, but there are lots of bad programmers.
--
Mike Nolan
htfoot@gmail.com <mailto:htfoot@gmail.com>

OK, I'm maybe responsible for this thread turning into a diatribe.  I
shouted at OP 'cause he shouted at us. My mistake, and I apologize.
I'm probably closer to Mike's "bad programmers" than I would care to
admit but fully believe software is a "people problem" more than most of
us realize.

It is kind of a purists fallacy. Point being if you could just write ASM
code it would be the best.

When in reality, a database is used not because it is the best technical
database, but is used by many people. Something that other developers can
pickup and use without reading a 200 page manual and study for a year on
end. Although maybe stuff would be better if everyone did that, on the
other hand might just be wasted effort.

You complain about no-SQL database but actually then advocate for it, by
saying SQL is sad. I find Postgres as a traditional RDB and has specific
use cases. If you compare that to Clickhouse which has a very different use
case. Don't compare timeseriesdb, because even that has limitations that
clickhouse surpasses at scale. Just an example of a no-SQL database.

If you do start a new database, let me know. I would like to see that in
action.

On Tue, Sep 14, 2021 at 9:20 AM Rob Sargent <robjsargent@gmail.com> wrote:

On 9/14/21 10:10 AM, Michael Nolan wrote:

I started programming in 1967, and over the last 50+ years I've programmed
in more languages than I would want to list. I spent a decade writing in
FORTRAN on a GA 18/30 (essentially a clone of the IBM 1130) with limited
memory space, so you had to write EFFICIENT code, something that is a bit
of a lost art these days. I also spent a decade writing in COBOL.

I've not found many tasks that I couldn't find a way to write in whatever
language I had available to write it in. There may be bad (or at least
inefficient) languages, but there are lots of bad programmers.
--
Mike Nolan
htfoot@gmail.com

OK, I'm maybe responsible for this thread turning into a diatribe. I
shouted at OP 'cause he shouted at us. My mistake, and I apologize.
I'm probably closer to Mike's "bad programmers" than I would care to admit
but fully believe software is a "people problem" more than most of us
realize.

--
Scottix@Gmail.com

I didn't start in 1967, but 1984, I'm in agreement with the bad
programmers premise. Since the beginning there have always been

lots of languages. It is my opinion, the more languages and concepts you
know the better your success on the project.

Heck I didn't use triggers till late 90's, funny thing I have a PICK
project right now.. too much fun

This is a subject which is important to me, but I find discussing it
often goes poorly. Most people (not necessarily those on this list)
don't distinguish between SQL and the relational model. When you
criticize SQL the language, people tend to defend relational
databases; when you praise relational databases, people have a
visceral reaction to SQL.

There also seems to be a divide between people who use languages to
express their thoughts, with the expectation that their thoughts will
be implemented, and those who regard a language merely as an interface
for manipulating an underlying system. I think there's a lot of good
to be said of workmen who get the job done without complaining about
their tools. But in the big picture, it seems to me that all the
progress we've made with computers has been a matter of improving the
toolchain. The CPU is, after all, an underlying system, and there's
nothing you couldn't get done with assembler (or just machine code).
If you were smart enough, and had enough time.

The problem is, tools tend to impose an "IQ tax": thought spent on
managing the tool is thought not spent on solving the problem. I tend
to be stingy about paying that; I'm not smart enough, and I don't have
enough time.

Merlin's point about algebraic syntax fits well, here. Once you're
used to it, (x ∩ y) imposes less of a cognitive load than SELECT *
FROM x INTERSECT SELECT * FROM y. You can see how that scales, as
expressions get larger. There's a reason we no longer try to make
programming languages look like English, or other natural languages,
however reasonable it might have seemed in the 1970s.

And then there are very simple things I can't say reasonably, like
"SELECT * EXCEPT col_3", or "Tell me how many nulls are in each
column." Naturally, I can get these done; but the gap between what
was required to explain the goal and what is required to accomplish it
is much too large.

So, the affection I have for SQL is due to it being a gateway to a
great idea; my frustration is that it's a bottleneck in getting to
that same idea.

On Tue, Sep 14, 2021 at 12:20 PM Rob Sargent <robjsargent@gmail.com> wrote:

On 9/14/21 10:10 AM, Michael Nolan wrote:

I started programming in 1967, and over the last 50+ years I've programmed in more languages than I would want to list. I spent a decade writing in FORTRAN on a GA 18/30 (essentially a clone of the IBM 1130) with limited memory space, so you had to write EFFICIENT code, something that is a bit of a lost art these days. I also spent a decade writing in COBOL.

I've not found many tasks that I couldn't find a way to write in whatever language I had available to write it in. There may be bad (or at least inefficient) languages, but there are lots of bad programmers.
--
Mike Nolan
htfoot@gmail.com

OK, I'm maybe responsible for this thread turning into a diatribe. I shouted at OP 'cause he shouted at us. My mistake, and I apologize.
I'm probably closer to Mike's "bad programmers" than I would care to admit but fully believe software is a "people problem" more than most of us realize.

--
Ray Brinzer

On Tue, Sep 14, 2021 at 1:54 PM Raymond Brinzer <ray.brinzer@gmail.com>
wrote:

So, the affection I have for SQL is due to it being a gateway to a
great idea; my frustration is that it's a bottleneck in getting to
that same idea.

I have the opposite perspective. As a dev/manager, SQL is much more
powerful at getting data storage from abstract concept, into a usable
structure, than any programming language I've had the (mis)fortune of
using. I've long since lost count of the massive volume of other people's
code (especially ORMs) I've removed and replaced by updating SQL statements
to do all the logic, and return me exactly what I want. And typically this
also comes with a (sometimes massive) performance gain.

I've managed many a programmer that would complain that SQL is too hard and
they don't want to learn it, but had no problem spending days learning the
ORM of the month that "saves them time" and writing complex inscrutable
monstrosities with them.

Could SQL be better? Absolutely. But in terms of bang-for-my-buck, I
feel learning SQL has saved me more clock-time, and improved my
productivity/value probably more than any other individual language in my
career.

On Tue, Sep 14, 2021 at 2:41 PM Brian Dunavant <dunavant@gmail.com> wrote:

I have the opposite perspective. As a dev/manager, SQL is much more powerful at getting data storage from abstract concept, into a usable structure, than any programming language I've had the (mis)fortune of using. I've long since lost count of the massive volume of other people's code (especially ORMs) I've removed and replaced by updating SQL statements to do all the logic, and return me exactly what I want. And typically this also comes with a (sometimes massive) performance gain.

I've managed many a programmer that would complain that SQL is too hard and they don't want to learn it, but had no problem spending days learning the ORM of the month that "saves them time" and writing complex inscrutable monstrosities with them.

Could SQL be better? Absolutely. But in terms of bang-for-my-buck, I feel learning SQL has saved me more clock-time, and improved my productivity/value probably more than any other individual language in my career.

Your experience doesn't surprise me at all. Sure; it's better than
the alternatives. An ORM can be a net benefit if you're doing simple
things, but the more complex the query, the more it starts to feel
like you're trying to have a serious conversation through a bad
translator. This encourages programmers to keep queries simple, treat
the database as a big scratchpad, and do all the processing in code.
This easily turns into "reinventing the wheel, badly".

I would argue, though, that the programmers aren't completely wrong.
A good programmer strives for clarity, expressing ideas simply and
naturally, and avoiding repetition; SQL isn't good for that. But
papering over the problem on the software side isn't the solution.

I'd just emphasize our agreement: SQL (or another query language,
sitting in the same niche) could be better. So it should be.

--
Ray Brinzer

Replies in-line

On 9/14/21 01:51, Guyren Howe wrote:

They are making a decent decision. SQL is a *fucking terrible*
language, which I don’t blame them for not wanting to learn.

Based on what criteria?

The whole industry, programming languages, infrastructure, everything
would have developed differently if relations were a natural,
pleasurable thing to use in any programming language. Like an Array,
or a Hash.

Thee is nothing natural about either relations or arrays and
hashes/dictionaries. Relations are pretty literal implementation of the
basic set theory. Having a decent understanding of the basic set theory
is a condition  for understanding SQL. Now, we can discuss whether a
language implementing a mathematical theory is "good" or "bad", whatever
the meaning of "good" or "bad" in the given context. Historically, SQL
is a good fit for the banking business and accounting and that is why it
is still around.

Another misconception about SQL is treating it as a general purpose
programming language. SQL is data description language, nothing more,
nothing less. It doesn't need loops, arrays, hashes or subroutines, its
principal purpose is to describe a subset of data. Without SQL you would
have to read all the data and filter the unnecessary stuff yourself.
Furthermore, part of SQL are so called "ACID requirements". Transaction
can only see the data that was committed before the transaction has
begun. Implementing ACID takes a lot of code, that's what MVCC is all
about. However, that too has its origins in accounting. You cannot pay
the money you don't have. And the last thing about SQL is transaction
management. Without relational databases and SQL, you would need a
proprietary transaction manager, just like MongoDB. And MongoDB has a
complex proprietary transaction manager and is losing market share. So,
to recapitulate:

* Declarative subset definition
* ACID consistency
* Transaction management
* Ideal fit for accounting.

That is why SQL is still around. And that is why we all live in a yellow
subroutine (this reference is not for the millennials or younger).

--
I'll speak the key, the whole key and nothing but the key, so help me Codd.
Mladen Gogala
Database Consultant
Tel: (347) 321-1217
https://dbwhisperer.wordpress.com