I have started a new thread for two issues:
-- CASCADE
-- Benchmark Considerations
that were raised in this thread/post:
https://groups.google.com/g/comp.databases.theory/c/pCxJKwKMsgc/m/ZPCw002wAQAJ

> On Monday, 14 June 2021 at 01:13:54 UTC+10, Nicola wrote:
>
> > Note that one should be used to that [DELETE+INSERT instead of UPDATE] because UPDATE any Key component
> > is not permitted, it must be DELETE+INSERT, in a Transaction of
> > course, that moves the entire hierarchy belonging to the Key.
>
> That is because in your experience, cascading updates (UPDATE ...
> CASCADE), are inefficient, right?

“Inefficient” is putting it mildly, it misses the relevant consideration.

CASCADE is simply not allowed, it isn’t even available on commercial SQL platforms.
Why not ? God help me, an explanation is demanded.
Because high-end SQL Platforms are heavily OLTP oriented.
And CASCADE would be a disaster in an OLTP context.

It is not about my experience, it is about knowledge of the server; server resources; maximising concurrency; minimising contention. Overall configuration and monitoring, first as a general task on every server, and then second, for the particular app and database.

>>
Theoreticians in this space, in their total abdication of responsibility, say that the server is a black box, and that they should not concern themselves with *ANY* implementation concerns. That is like saying the engineer of the car should not concern himself with the exercise of driving. It is the asylum, where total incapacity is allowed, propagated to “science”. It is acceptable only for classroom exercises, utterly bankrupt outside the classroom.

Implementation concerns are the everyday fare for a DBA, and certainly the concerns of a Data Modeller in the latter stages. The black box is a physical reality, it is grey and blue, it fits into a rack in the computer room. All requests to the server pass through it. It is not an abstraction in the classroom that can be ignored.

>>>>
From this post:
__ https://groups.google.com/g/comp.databases.theory/c/Uwc_w8HbBfw/m/trHkR1F8Mx8J

Take “business rule” in the referenced post as your “cascade” here. Cut-paste:

“
Ok, that means you do not understand the world of implementation.

1. On one side, where the business gives us "business rules", they are not to be taken as implementation imperatives. If taken as such, we would be merely clerks, implementing their requirements, without using the skills that they hired us for. Eg. we would implement a "business transaction" that updated six million rows, that hung the users up for 15 minutes in the middle of the day, and we would take no responsibility, because the business "told us to do it".

1.a Obviously, we do not do that. We exercise the skills we were hired for. Part of which is to implement OLTP Standard-compliant transactions. We do not view the business requirements as imperatives, we view them as initial requirement statements. We work back and forth, such that the requirements are modified, then accepted, and then implemented, such that they do not crash the system; such that the database does not have circular references; etc; etc; etc.

1.b So the example "business transaction" would be converted into a batch job that runs in a loop and executes six million OLTP Standard-compliant single-row transactions. The batch job keeps track of its position; is restartable; etc. So the business gets the requirement they want, but not in the METHOD that they initially stated it. Ie. Just tell me what you want, don't tell me how to do it.

1.c On this one side, in no case is a business rule to be taken as an imperative.
“
<<<<

>>>>
Also this post:
__ https://groups.google.com/g/comp.databases.theory/c/qqmnhu036FQ/m/RLh9D5Ue1kUJ
please read this section:
__ III - Batch Transaction
<<<<

The most contentious object in the database is the Transaction Log (or its equivalent on MVCC systems, and by any name, in any location).
__ On “2PL” systems, it is a small, hot object, and we try to keep it small (I am not giving you the entire science here, but there are clear, defined rules, such as OLTP Standards).
__ On MVCC systems, it is huge and spread across the entire file space, and the developers are unaware of it, thus trying to constrain its use is not even a remote possibility. Everyone is programmed to pretend that the database is single-user and that they are the single user (the Stonebraker insanity). MVCC does not have ACID.

Note, Functions do not have to be Transactional or NotTransactional, that is a stupid artefact of some pretend-sqls. In ACID, Transactions are declared by the caller, not by the called object.

By virtue of implementation Standards (Software Gems in particular, because we guarantees high concurrency, zero deadlocks), here OLTP Standards, the limit for number of rows affected in an OLTP Transaction is 100, for batch Transactions 500.

So think that out. Write an ACID Transaction (no I am not being silly, I realise you can’t on your non-SQL platform, so you have to think in SQL terms, for an SQL platform). A stored proc, that:
- navigates the *levels* of the Tree,
- and loops,
- executing max 100 INSERTs per BEGIN::COMMIT for the new Key,
- then executing max 100 DELETEs per BEGIN::COMMIT for the old Key.

Whereas CASCADE or the equivalent will update six million rows under the covers and hang up the database (prevent other updaters from progressing) for a week or two, this OLTP Transaction will execute in minutes, without hanging anyone else up.
<<

> I'd like to do some benchmarks.

Great idea. Excellent for obtaining experience in real world issues. But Whoa, that means you have to acquire knowledge of the physical implementation, both what you want to do in your database and how that translates into physical objects, as well as how your pretend-platform implements it. Eg. Indices; types of indices; Transaction Log or additional rows on every touched page plus a garbage collector. Etc, etc, etc.

*Benchmark* generally implies not one but two platforms, and a comparison. So I would plead that you obtain a “2PL” platform for the second. The commercial SQL Platforms all provide a “developer version” which is free and limited (eg. number of simultaneous users or max table size; etc). There is no point is comparing one MVCC monster with yet another MVCC monster, you will learn nothing in the benchmark category (except difference in internals of the two freeware suites).

*Benchmark* strongly implies an ability to monitor the server, all metrics that pertain to performance (in general), and the particular metrics that pertain to the particular benchmark. Freeware has no such thing, so you will be working in the dark, with little idea re what is happening under the covers.

For freeware & Oracle, due to not having a Server Architecture, and instead deploying hundreds or thousands of programs running on Unix, the first avenue of “performance monitoring” and even “problem diagnosis”, is via Unix monitoring:
-- top
-- vmstat

For contrast, take a glimpse of what is available in commercial SQLs. The server is a genuine Server Architecture, so the monitoring regards internal metrics, and very little of Unix (set up correctly once, and forgotten).

0. Monitoring products
Additional licence fee, not shown. Feel free to search the internet.

1. Raw Stats: text, various forms. Free.
Voluminous, not shown. Can be ascertained from the following.

2. Raw Stats Formatted, especially for various types of comparison.
Uses a simple script to produce [2] from [1].
Here, I fixed an error that the SAN configuration boys made. the comparison is Before::After the change: virtually the same as a benchmark, but this is monitoring the production server at the largest teaching hospital in America. With permission of course.

__ https://www.softwaregems.com.au/Documents/Article/Sysmon%20Processor/Sysmon%20Processor%20Eg%20Date%20Delta.pdf

“Div” is the division of the metric by the relevant denominator.

“Dlt” is the delta, between that column-of-stats and the first column-of-stats, as a percentage, with {-|+|x}.

“RUt” is Resource Utilisation, shown for unit resources, as a percentage of the ResourceGroup (which is shown above the unit resources)

“Engine” is a single Unix Process. The Sybase ASE server comprises a grand total of FOUR Engines in this instance, and serves hundreds of active connections (doctors; nurses; medical images; client histories; etc). In contrast, freeware and Oracle would have hundreds of Unix Processes and no concept of Engine.

The entire server config is set up by me, not the local DBA, whom I support.. That means the resource allocation strategy is proprietary, so all the resources are shown, but the names [eg. cache names] have been obfuscated.
- IIRC the freeware, and Oracle, have no concept of a cache, let alone control of it.
- Disks [SAN objects] are the slowest link in the chain, and the hardest to change, so they are set up correctly, once and for all.
--- Until some idiot in the SAN team made an undeclared change, that had a negative effect, that I had to diagnose, without knowledge of the change.

This is a “2PL” server, you may find the Lock Manager stats interesting.

Likewise the Transaction section.
The app is not a very good one, it is stupid, because Parallelism has to be turned off, in order for the app to work. It is not strict OLTP, but good enough in the sense that it does not cause problems. Its ACID profile is hopeless. Nevertheless, via resource management, I have managed to reduce problems and virtually eliminate deadlocks.

On 2021-06-14, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> So think that out. Write an ACID Transaction (no I am not being
> silly, I realise you can’t on your non-SQL platform, so you have to
> think in SQL terms, for an SQL platform). A stored proc, that:
> - navigates the *levels* of the Tree,
> - and loops,
> - executing max 100 INSERTs per BEGIN::COMMIT for the new Key,

Let me call this T1...

> - then executing max 100 DELETEs per BEGIN::COMMIT for the old Key.

....and this T2.

Why is the state after T1 has committed and before T2 starts considered
valid? Wouldn't a query performed between T1 and T2 get an inconsistent
view of the data?

Another question: if T1 requires 200 INSERTs instead of 100, you would
split it in two. Again, how can you consider the intermediate state
(after the first 100 INSERTs, but before the remaining ones) valid?

Nicola

Nicola

> On Tuesday, 15 June 2021 at 03:39:09 UTC+10, Nicola wrote:
> > On 2021-06-14, Derek Ignatius Asirvadem wrote:
> > So think that out. Write an ACID Transaction (no I am not being
> > silly, I realise you can’t on your non-SQL platform, so you have to
> > think in SQL terms, for an SQL platform). A stored proc, that:
> > - navigates the *levels* of the Tree,
> > - and loops,
> > - executing max 100 INSERTs per BEGIN::COMMIT for the new Key,
> Let me call this T1...
> > - then executing max 100 DELETEs per BEGIN::COMMIT for the old Key.
> ...and this T2.

We can call it T1; T2, but we don’t want to confuse that with a normal ACID Transaction T1; T2, because it is not, it is a Batch Transaction. Whereas in ACID, we are dealing with {Add|Drop|Mod} a logical Atom of data (RM-compliant db = coincident with Logical Atom; non-compliant db = does not have Logical Atom), here we are moving an entire tree, of many levels, therefore State in the ACID sense will not apply.

We may be better off calling it B1; B2.

It is [Relational] Set Processing (much faster), not row processing, with the Set Size limited to chunks of 100. You can’t do this with say a CURSOR.

The State that is relevant is the State of the whole tree, either the whole tree is in the OldKey xor the whole tree is in the NewKey.

Lock the OldKey tree for the duration. Based on your platform, that will require different verbs (if it has a verb for locking). Obviously, only the single row at the top needs to be locked. Here you are performing the ISOLATION manually. Unlocking OldKey is not necessary, because at the end all OldKey rows are deleted.

If you have implemented *Optimistic Locking* (a well-known component of the OLTP Standard, which affords high concurrency), it makes life much easier. Ie. I don’t have to use a LOCK (whatever variant) command, I am locking Logically (data level), not physically (server level).

Because it is a first cut, you don’t have to implement this capability, it is an FYI for understanding. We write such batch transactions as recoverable. That means we can track precisely where the *move* failed, or stated otherwise, how much the NewKey tree succeeded. You need to be able to code SQL IN and NOT IN on large sets, with their composite Keys, with confidence re speed (you can’t in Oracle), to be able to do this.

One decision you have to make is, how much of the NewKey tree is visible, that is NOT-ISOLATED. Our guideline (within the standard) is:
- for financial or audit-required data, lock the single row at the top of the NewKey tree,
- for the rest, allow it (partial tree, in progress, level-at-a-time) to be visible

> Why is the state after T1 has committed and before T2 starts considered
> valid? Wouldn't a query performed between T1 and T2 get an inconsistent
> view of the data?

The [available] data is never inconsistent. An user gets whatever he queries, from either a whole OldKey tree xor a whole NewKey tree (partial levels for non-audit data ... which is changing by the millisecond, as accepted by the users/auditors).

> Another question: if T1 requires 200 INSERTs instead of 100, you would
> split it in two. Again, how can you consider the intermediate state
> (after the first 100 INSERTs, but before the remaining ones) valid?

I think that is mostly answered above.

The “split” is not an act per se. In Sybase or MS/SQL, the command to limit the rows-affected-by-verb is:
__ SET ROWCOUNT n
which operates until reset:
__ SET ROWCOUNT 0
Just exec that before your first WHILE loop, and clear it after.

======================== Standard: Two Options =======================
Upon reading my OP, I noticed a bit of non-clarity, which I will clear here..

Without the Standard, let’s say for a “good” database (ignoring other errors), you would CASCADE updates to a Key (mickey mouse non-server), or UPDATE Key, and suffer horrendous contention and blow the Transaction Log.

--------------------------------------------------------------
-- OLTP Standard • Batch Transaction --
--------------------------------------------------------------

With the Standard, which is what we deliver as minimum, all such contention is eliminated. Described above. Very Fast, meaning visibility of the partially-built NewKey tree is not really an issue. But it requires thoughtful coding.

Once implemented, it can be enhanced even further (not described here).

----------------------------------
-- Transaction Loop --
----------------------------------

But many times, we do not have a /Replace the DB/RFS with a real RBD/ directive, we are only there to perform a small or large consulting assignment. In any case, that involves education. The result of such education is, the developers with the new awareness of what happens under the covers, want to elevate their mess to less-mess. That is, they want to do anything that will reduce the contention problems they are suffering: they want to stop using UPDATE Key.

Where they have honest ACID Transactions for their normal operations (eg. OrderSaleItem_Add_tr), that is relatively easy. (Where they don’t, the mess is a Date/Darwen/Stonebraker pig sty, and has to be replaced, not fixed-in-place.)

Write a much simpler set of loops, and exec the existing Transaction sprocs at each level. This means the rows are moved from OldKey to NewKey one-Transaction-at-a-time. Eliminates one set of contention problems immediately.. It is slow, but only when compared to genuine Batch Transactions (which they don’t have). It is not a half-way point in reaching compliance with the Standard, but it is easy to do, and relief is immediate.

That is what I meant in [1.b] in the OP, and in the linked post (which has a different context):

> > On 2021-06-14, Derek Ignatius Asirvadem wrote:
> >
> > 1.b So the example "business transaction" would be converted into a batch job that runs in a loop and executes six million OLTP Standard-compliant single-row transactions.

Let’s put it this way. Some developers are responsible, proud of their work. When they obtain education, they have a natural urge to fix-up their code to eliminate the causes of their contention problems (effects) that they are now-aware of. They want to be able to say /my code complies with Derek’s Standard, it is non-contentious/ or /my code isn’t the Standard, but it is the least contentious that we can manage right now/.

One cannot move from this category to the OLTP Standard-compliant category, without at least implementing Optimistic Locking; etc. If one does implement OLTP Standard, it is best to go the whole hog: rewrite the database for full /RM/ and ACID and Optimistic Locking; write a full set of ACID & Batch Transactions; etc. No half-measures.

Cheers
Derek

Nicola

> On Tuesday, 15 June 2021 at 08:08:25 UTC+10, Derek Ignatius Asirvadem wrote:
>
> > On Tuesday, 15 June 2021 at 03:39:09 UTC+10, Nicola wrote:
> >
> > > On 2021-06-14, Derek Ignatius Asirvadem wrote:
> > >
> > > So think that out. Write an ACID Transaction (no I am not being
> > > silly, I realise you can’t on your non-SQL platform, so you have to
> > > think in SQL terms, for an SQL platform).

A Caveat that is really important. In my years of dealing with developers who are used to the Stonebraker/MVCC mindset, it is clear that they do not grasp ACID. They are so used to the Oracle/PusGres *redefinition* of ACID (fraud), to fit their MVCC mindset, for the purpose of making their MVCC somewhat palatable, that they cannot grasp that they do not know ACID. That state of ignorance allows them to argue that MVCC is comparable to “ACID”. MVCC is not comparable to ACID, it is the polar opposite..

Flat and absolute declaration: MVCC systems cannot provide ACID, in particular the ISOLATION LEVELs. The consequence is, MVCC systems cannot provide any level of genuine OLTP.

So, even though the current line of questioning is about Batch Transactions, it may be hindered by an incorrect understanding of ACID.

> > > Write an ACID Transaction

Sorry, no, it is a Batch Transaction with no regard to ACID.

> > > A stored proc, that:
> > > - navigates the *levels* of the Tree,
> > > - and loops,
> > > - executing max 100 INSERTs per BEGIN::COMMIT for the new Key,
> >
> > Let me call this T1...
> >
> > > - then executing max 100 DELETEs per BEGIN::COMMIT for the old Key.
> >
> > ...and this T2.
>
> We can call it T1; T2, but we don’t want to confuse that with a normal ACID Transaction T1; T2, because it is not, it is a Batch Transaction.. Whereas in ACID, we are dealing with {Add|Drop|Mod} a logical Atom of data (RM-compliant db = coincident with Logical Atom; non-compliant db = does not have Logical Atom), here we are moving an entire tree, of many levels, therefore State in the ACID sense will not apply.
>
> We may be better off calling it B1; B2.

Whereas T1; T2 are ACID Transactions, and State means database state; Consistency per all Constraints, ie. Logically Atomic, B1: B2 are physical chunks (delimited by an arbitrary number), in a physical *move* operation. It is harmless ACID-wise because the OldKey tree is ACID-compliant, and we are not affecting that in any way, we are faithfully carrying that over into the NewKey tree.

Such operations are not foreign to a DBA or developer, because they are used to *moving* large chunks of data in their normal day-to-day work. Eg. moving a logical xor physical subset of a table from Production to Development for test purposes. For logical (Atomic) purposes, that means not one table but a set of tables (a branch of a tree). Such operations may be new to some readers.

Cheers
Derek

Nicola

> On Monday, 14 June 2021 at 14:35:13 UTC+10, Derek Ignatius Asirvadem wrote:
>
> > On Monday, 14 June 2021 at 01:13:54 UTC+10, Nicola wrote:
> >
> > I'd like to do some benchmarks.
>
> Great idea. Excellent ...
>
> *Benchmark* strongly implies an ability to monitor the server, all metrics that pertain to performance (in general), and the particular metrics that pertain to the particular benchmark. Freeware has no such thing, so you will be working in the dark, with little idea re what is happening under the covers.

> For contrast, take a glimpse of what is available in commercial SQLs.
>
> 0. Monitoring products
> Additional licence fee, not shown. Feel free to search the internet.

There are quite a few.

0.a
Bradmark is probably the best of breed. Great graphical representation with drill-down, etc. Note that we have had these for thirty years, ten years before the open source anti-market produced you-get-what-you-pay-for “database systems”.

__ https://www.bradmark.com/products/survSybaseASE.html

AFAIK, none of the players provide support for freeware. This is not to say that there is not a market (there is, there are suckers born every minute). The reason is, there is nothing, no performance stats produced, to monitor. Worse, whatever does exist, keeps changing with every major version. Just think about how “transaction” and “transactional” issues have changed in the last ten years, and we are still nowhere near ACID.

0.b
Sybase used to have a full-blown Monitor Server, that sat on the same box as the DB server. Heavy duty like you would not believe. Perfect for benchmarks. Slowly made obsolete as 3P products gained market share.

> 1. Raw Stats: text, various forms. Free.
> Voluminous, not shown. Can be ascertained from the following [which are summaries].

Generally two types:
1.a
Internal counters, reported as requested (eg. 24 x 60 mins), zero overhead. This has been available from the beginning, and this is what I use to produce [2].

1.b
Monitoring & Diagnostic Access. A database in the server, that collects [1..a] and exposes them as Relational tables. Overhead is 5-15% depending on what is collected (configurable). Eg. collection of execuing SQL can be heavy.

> 2. Raw Stats Formatted, especially for various types of comparison.
>
> __ https://www.softwaregems.com.au/Documents/Article/Sysmon%20Processor/Sysmon%20Processor%20Eg%20Date%20Delta.pdf

I have given mine for comparison. Do you have a link or reference for monitoring PusGres ?

> “Dlt” is the delta, between that column-of-stats and the first column-of-stats, as a percentage, with {-|+|x}.

For a quick overview, just scan that column: existence of a double-digit value means the metric is worth examination; positive/negative is good/bad thing, depending on the metric of course.

“Selection”
At the top of the page. These are selected “Key Performance Indicators” or an executive summary.

Note that the load is identical (two Mondays chosen to ensure that). But the activity within the server is quite different. The first column shows the activity due to the SAN fault, the second shows the activity after the correction:
__ Read Locks reduced by 91%
__ Server/Context Switch reduced by 21%
__ Unix/Context Switch reduced by 24%
__ and of course far less CPU usage, at both levels

> For freeware & Oracle, due to not having a Server Architecture, and instead deploying hundreds or thousands of programs running on Unix, the first avenue of “performance monitoring” and even “problem diagnosis”, is via Unix monitoring:
> -- top
> -- vmstat

“Host System”
The unix/vmstat metrics are at the bottom of the page.

Sybase is a dedicated server, meaning that it is designed for tight integration (even binding) with the o/s and hardware, and nothing else should be run on the box. This box additionally hosts runs a small MySQL and a few other small things, and the degree to which they allow me (Level 3 support) to tightly integrate with the o/s is limited. Point being, performance is reasonable, but nowhere near best possible for the box. Eg. I can improve throughput significantly, and of course I would cancel the Monitor db.

“Selection”
At the top of the page. These are selected “Key Performance Indicators” or an executive summary.

> “Engine” is a single Unix Process. The Sybase ASE server comprises a grand total of FOUR Engines in this instance, and serves hundreds of active connections (doctors; nurses; medical images; client histories; etc). In contrast, freeware and Oracle would have hundreds of Unix Processes and no concept of Engine.

For those who labour over performance tuning of Oracle or PusGres, because it lacks a genuine Architecture diagram, here it is. Please don’t say that I only help the top end, that I don’t help the bottom-feeders. They are identical, just substitute the Oracle component names with the PusGres equivalents:

__ https://www.softwaregems.com.au/Documents/Article/Oracle%20Circus/Oracle%20vs%20Sybase.pdf

Cheers
Derek

On 2021-06-14, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> Nicola
>
>> On Tuesday, 15 June 2021 at 03:39:09 UTC+10, Nicola wrote:
> The State that is relevant is the State of the whole tree, either the
> whole tree is in the OldKey xor the whole tree is in the NewKey.

Would you mind letting me understand with an example (I'd like to grasp
the overall idea, which I have not yet; I am not asking you to reveal
the secret recipes of your shop) from this document:

https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf

Suppose that, for some reason, one needs to update one HotelChain,
changing the value of HotelChain from ABC to H-ABC. If I understand
correctly, you would start by (optimistically) locking the record with
HotelChain = ABC.

What would you do next? Insert the new record with key H-ABC into
HotelChain, then insert zero or more records referencing H-ABC into
Hotel, then into HotelNetwork, ... (up to 100 inserts), commit; more
inserts (up to 100), commit, etc.?

How would you delete the old records then? With transactions of up to
100 DELETEs each, starting from the bottom of the hierarchy and
navigating the hierarchy up?

> If you have implemented *Optimistic Locking*

It's interesting that you mention optimistic locking (have you ever
mentioned in before in this group?), because my understanding was that
you believe that the only correct way to control concurrency was strict
pessimistic locking (strict 2PL). This may have been a misunderstanding
on my part.

Nicola

Nicola

> On Wednesday, 16 June 2021 at 18:57:23 UTC+10, Nicola wrote:
> > On 2021-06-14, Derek Ignatius Asirvadem wrote:
> >
> >> On Tuesday, 15 June 2021 at 03:39:09 UTC+10, Nicola wrote:
> > The State that is relevant is the State of the whole tree, either the
> > whole tree is in the OldKey xor the whole tree is in the NewKey.
>
> Would you mind letting me understand with an example (I'd like to grasp
> the overall idea, which I have not yet; I am not asking you to reveal
> the secret recipes of your shop) from this document:
>
> https://www.softwaregems.com.au/Documents/Student_Resolutions_2020/Giannis/RoomMonitor%20DM%20V0_51.pdf

They are not secrets in the sense that they are natural progressions of the /RM/, due to a deeper understanding of it. I am sure other serious boutique consulting houses have done at least some of those things. Perhaps not with proper labels and standardisation.

But sure, I answer questions. As deeply as the person is capable of receiving. Which on this forum is only you.

However, this subject is something that absolutely every single programmer should know, not just those developing code to run against an OLTP RDB or “OLTP” RFS. It is a programming construct, a Batch Job. Running against an OLTP database. The DayEnd or MonthEnd procedure at a bank. Move OldKey to NewKey. Some minor deployment requirements, but the construct is the same. I am happy to give full details in public. Particularly because I am shocked that SQL coders do not know this.

The OLTP database and particularly its API (set of ACID Transactions) are already designed for low contention; high concurrency. Think: cars running along the streets of a city. A transaction that affects 10K rows is like a train running along surface streets (no rail tracks, we have one TransactionLog file that cars and trains; boats and planes, all fight over), across the city. That is what contention is. It requires perceiving the whole, as well as the parts. (You guys are trained to think about parts only, and one-at-a-time.)

Cars are allowed max 1 trailer; trucks max 16 trailers; trains max 100 trailers.

> Suppose that, for some reason, one needs to update one HotelChain,
> changing the value of HotelChain from ABC to H-ABC. If I understand
> correctly, you would start by (optimistically) locking the record with
> HotelChain = ABC.

Yes.
And leave it locked for the duration, it will be the last deleted.

>>>>
> (optimistically)

Because I know that you guys do not understand Locking, let alone Optimistic Locking, note this this is not an act within that [generic, theoretical] framework. This is a data-level lock (if you have row locking, and a row locking command, you do not have ACID *and* you have broken a cardinal rule of OLTP).

If you have Soft Deletes, use that.
<<<<

> What would you do next? Insert the new record with key H-ABC into
> HotelChain, then insert zero or more records referencing H-ABC into
> Hotel, then into HotelNetwork, ... (up to 100 inserts), commit; more
> inserts (up to 100), commit, etc.?

Yes.
Code WHILE loops. One WHILE loop per level. SQL has INSERT-SELECT, it is Set Processing, very fast.

Limiting the affected set is simple, in MS or Sybase syntax:
__ SET ROWCOUNT { 0 | n }

Some people would code a cursor. It is pure laziness, same as CTEs. Massive overhead, and totally unnecessary. I don’t allow a cursor on my servers.

> How would you delete the old records then? With transactions of up to
> 100 DELETEs each, starting from the bottom of the hierarchy and
> navigating the hierarchy up?

Yes.
OldKey rows.

> > If you have implemented *Optimistic Locking*
>
> It's interesting that you mention optimistic locking (have you ever
> mentioned in before in this group?),

I don’t keep track of such details, so I can’t say.

Optimistic Locking is a science that existed from the early 1960’s, it is an essential part of OLTP. I have just Relationalised and SQLised it, and made it the OLTP Standard. Which in turn is an essential part of Open Architecture. There are components that have to be in the tables, and of course code components in every ACID Transaction. Science is truth, and truth does not change, it has not changed since 1973 [when I came to know of it]. I openly credit IBM Mainframe CICS/TCP (their transaction server, that you guys think is “batch”). That is Transaction Control Process.

Academia do not know that. So the imbecile Stonebraker read the IBM manuals and thought, with his great charisma, he could do better. So I know, from the bits I have read in this forum, that you guys have a fantasy version of “Optimistic Locking”.

> because my understanding was that
> you believe that the only correct way to control concurrency was strict
> pessimistic locking (strict 2PL). This may have been a misunderstanding
> on my part.

Well, I don’t have a problem with what you have picked up. The thing that needs to be understood is, you have picked up whatever I stated, but from your pre-existing mindset. First, noting the above ridiculous notion of “Optimistic Locking”, that consists of a few notions (not standards) from Optimistic Locking.

Second there is no such thing in reality as “pessimistic locking”. Or that “2PL” is “pessimistic locking”, it is absurd. I can only think that the idiots who said so, in those grand papers posed such a silly thing as a Straw Man argument, to make “2PL” systems look bad.

No. Optimistic locking has nothing to do with “2PL” locking. The former is a construct deployed in tables and ACID Transaction code. The latter is the resource contention resolution mechanism on the server, quite removed from the former.

MVCC is not optimistic. It may well be “optimistic” to those precious darlings who chant the mantra /Readers don't block writers, writers don't block readers/, which exists only in the imagination, not in reality, and certainly not in their precious MVCC database that keeps hanging up despite all the users, together now, chanting the mantra /Readers don't block writers, writers don't block readers/.

So no, coming from that mindset, due to the quite different notions attributed to terms (see how the pig poop eating freaks destroy science), reading my posts, which may not deal with the issue directly, you will have some incorrect understanding.

Yes, I confirm, the only valid contention resource resolution mechanism in an OLTP database, is Ordinary Locking (science unchanged since the 1960’s, steadily advanced in the chip design sector). Which is not pessimistic, no matter how much you call it that. Now you guys call that “2PL” or strict “2PL”. (I don’t know how strict has anything to do with it. Concerning locks and security, it is binary: either you have it or you don’t.) So for me, 1PL or “2PL” is strict, we can’t strictify it any further.

I may have been lazy here in the past, by not rejecting the term “2PL”. From the little I do know, it is yet another mad construct on the academic-only collective imagination. It is not a good idea to place that label onto me, because the definition is too loose and unscientific, and I do somewhat more than that. Properly explained, I would not use stupid terms such as “expanding” and “shrinking”, no, they are two code blocks. In this farcical “definition”:
__ https://en.wikipedia.org/wiki/Two-phase_locking
half those categories do not exist, even describing them is hysterical.

Eg. in “C2PL”, the imbecile acquires locks “before” the transaction, without realising that the only way to acquire locks is “INSIDE* a transaction. Cesspool.

Eg. 1PL [to the OLTP Standard] eliminates deadlocks, that is one declared guarantee. These freaks call it “2PL” and state that deadlocks can happen, and elsewhere that it causes deadlocks.

Eg. I will bet you, you guys can’t even define a deadlock, let alone guarantee preventing it.

Ok, according to the latrine, Sybase; DB2; and MS are “SS2PL”, not “S2PL”.

And of course 1PL is much more, more than the total collective imagination of “SS2PL” is among the academics.

On that wiki page, the particular odour of fæces is, that of a female pig, a sow with full teats. All 12 piglets wrote that one page together, right between those warm teats. It is just a fanciful write-up of various narrow views of Ordinary Locking, which are self-serving. “Scientifically” defined Straw Men, the purpose of which is to *AVOID* understanding Ordinary Locking.

Actually, it is an attack on ACID, because any description of Transaction that are not ACID, is anti-ACID. They don’t even mention it, because the entire page; all descriptions, fail the [A]tomic requirement. And thus they can write 100 papers about how MVCC is sooooo much better than those Straw Men. And continue denying reality. Pffft.

The only valid papers on Locking were written by IBM and Britton-Lee/Sybase engineers. Stonebraker is a fantasist, a cult leader, not a scientific person by any measure. Kool Aid and mantras. Maybe some chakra balancing. Believe me, his downward dog never got up.

Yes, so all the high-end servers are Ordinary Locking or “1PL” or “SS2PL”, whether it contains a database or not. Separately, on the database and Transaction code side, we implement Optimistic Locking, for OLTP, not because the server is OL/1PL/SS2PL.

On 2021-06-16, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
>> Suppose that, for some reason, one needs to update one HotelChain,
>> changing the value of HotelChain from ABC to H-ABC. If I understand
>> correctly, you would start by (optimistically) locking the record with
>> HotelChain = ABC.
>
> Yes.
> And leave it locked for the duration, it will be the last deleted.

>>>>>
>> (optimistically)
>
> Because I know that you guys do not understand Locking, let alone
> Optimistic Locking, note this this is not an act within that [generic,
> theoretical] framework. This is a data-level lock (if you have row
> locking, and a row locking command, you do not have ACID *and* you
> have broken a cardinal rule of OLTP).

Let me see whether we are on the same tune. While you are performing the
batch transaction we are talking about, another transaction may also
attempt to update the record with HotelChain = ABC (and specifically
update the HotelChain field). Is it correct that this second transaction
will be allowed to perform the update, and that first (batch
transaction) will detect at the next commit that the "top" row has been
overwritten, and rollback (at least rollback the running ACID
transaction)?

Because that is my understanding of "optimistic". But the rest of your
post, (and the remark above) make me doubt that we agree on this
meaning.

Except from the point above, "batch delete+insert" is clear enough.

Nicola

Nicola

> On Thursday, 17 June 2021 at 05:45:09 UTC+10, Nicola wrote:
> > On 2021-06-16, Derek Ignatius Asirvadem wrote:
> >> Suppose that, for some reason, one needs to update one HotelChain,
> >> changing the value of HotelChain from ABC to H-ABC. If I understand
> >> correctly, you would start by (optimistically) locking the record with
> >> HotelChain = ABC.
> >
> > Yes.
> > And leave it locked for the duration, it will be the last deleted.
>
> >>>>>
> >> (optimistically)
> >
> > Because I know that you guys do not understand Locking, let alone
> > Optimistic Locking, note this this is not an act within that [generic,
> > theoretical] framework. This is a data-level lock (if you have row
> > locking, and a row locking command, you do not have ACID *and* you
> > have broken a cardinal rule of OLTP).
>
> Let me see whether we are on the same tune. While you are performing the
> batch transaction we are talking about, another transaction may also
> attempt to update the record with HotelChain = ABC (and specifically
> update the HotelChain field). Is it correct that this second transaction
> will be allowed to perform the update, and that first (batch
> transaction) will detect at the next commit that the "top" row has been
> overwritten, and rollback (at least rollback the running ACID
> transaction)?
>
> Because that is my understanding of "optimistic". But the rest of your
> post, (and the remark above) make me doubt that we agree on this
> meaning.

No, that is not correct.

Yes, you do *not* have a correct understanding of Optimistic Locking, none of the academics do, they have only the Stonebraker MVCC Kool Aid, and they have been drinking it and circle-jerking while chanting the *Mantra* for decades. As I have tried to explain, Optimistic Locking is completely different to the Straw Man that academics erect as “optimistic locking”. This point (your question) has nothing to do with Optimistic Locking.

It has to do with how to construct a Batch Transaction (MonthEnd Procedure at a bank or Move OldKey to NewKey).

>>>>
1. Lock top-most <Table>[ <OldKey> ]
For the duration of the Batch Transaction.
<<<<

>>>>
You need some column that indicates a data-level lock. Standard columns include CreatedDtm; UpdatedDtm; etc, that can be used for this purpose [eg. set CreatedDtm to TODAY+1 or other fixed value [it is temporal, yes] ). You may have IsDeleted to support SoftDeletes. Let’s use the easy one, and say you have Soft Deletes. For clarity, you need a condition [ IsLocked ], and you implement that as indicated in one of the above columns, if not, you have a column [ IsLocked ].

The fact that there is such a column, and that contention is understood and consciously minimised, *is* part of Optimistic Locking, part of the OLTP Standard.
<<<<

But this is not an ACID Transaction, it is a Batch Transaction. Of course, we use the column that is there for OLTP purposes, for the now Batch purpose.

In this case, that means:
-- UPDATE HotelChain.IsDeleted[ ABC ] = 1
---- WHERE HotelChain.IsDeleted[ ABC ] = 0
or
-- UPDATE HotelChain.IsLocked[ ABC ] = 1
---- WHERE HotelChain.IsLocked[ ABC ] = 0

That prevents both [a] and [b] in this:
> another transaction may also attempt to
> (a) update the record with HotelChain = ABC
> (b) (and specifically update the HotelChain field)

(Which row is meant by “record” is not clear, but it doesn’t matter for the answer: all rows from top to bottom that belong to [ ABC ] are locked.)

Separate point. That “another transaction” sounds suspiciously like another instance of the same transaction, using the same key. That doesn’t happen in the real world because the person who administers the first instance of changing ABC is one and the same person who administers the second instance of changing ABC. This sort of insanity is /also/ prevented by proper GUI design (a Tree with no circular references).

It also prevents:
-- UPDATE Hotel.NumStar[]
or
-- UPDATE HotelRoomType.Description[]

--------

To be clear, this is answering your question, which is you trying to understand Batch Transaction structure in an OLTP environment, (a) from the bottom up, and (b) with the known-to-be-straw-man concept of Optimistic Locking. This is *not* answering the question /what is Optimistic Locking/, which would be a top-down lecture, and include the considerations for genuine OLTP.

Cheers
Derek

Nicola

> On Thursday, 17 June 2021 at 08:36:12 UTC+10, Derek Ignatius Asirvadem wrote:

--------------------------
-- 1 Correction --
--------------------------

> In this case, that means:
> -- UPDATE HotelChain.IsDeleted[ ABC ] = 1
> ---- WHERE HotelChain.IsDeleted[ ABC ] = 0
> or
> -- UPDATE HotelChain.IsLocked[ ABC ] = 1
> ---- WHERE HotelChain.IsLocked[ ABC ] = 0
>
> ...
>
> It also prevents:
> -- UPDATE Hotel.NumStar[]
> or
> -- UPDATE HotelRoomType.Description[]

Mistake, sorry, that should be:
It also prevents any Transaction containing:
-- UPDATE Hotel.NumStar[ ABC ]
or
-- UPDATE HotelRoomType.Description[ ABC ]
or
-- INSERT <AnyTableInHierarchy>[ ABC ]
or
-- DELETE <AnyTableInHierarchy>[ ABC ]

----------------------------
-- 2 Impediment --
----------------------------

> Yes, you do *not* have a correct understanding of Optimistic Locking, none of the academics do, they have only the Stonebraker MVCC Kool Aid, and they have been drinking it and circle-jerking while chanting the *Mantra* for decades. As I have tried to explain, Optimistic Locking is completely different to the Straw Man that academics erect as “optimistic locking”. This point (your question) has nothing to do with Optimistic Locking.

The term Optimistic Locking and the method (Standard) that it defines, is IBM [360] CICS/TCP from the early 1960’s. That is science. That is truth that does not change.

Any use of the term to mean anything other than that, is fraudulent; dishonest conduct; anti-science.

Which filth Stonebraker; Ellison; Date; Darwen; Fagin; etc, all engage in, in their suppression of the /RM/ and their promotion of RFS as “Relational”, their suppression of SQL and their promotion of non-sql as “SQL”.

You can’t understand
__ Optimistic Locking
(the OLTP requirement; two non-server deployments) while holding on to any shred of the “optimistic locking” Straw Man that is heavily established in academia over the decades.

You can’t understand
__ server-level Locking
or
__ data-level Locking
while holding on to any shred of the “2PL”; “S2PL”; “C2PL”; SS2PL”; etc, fantasies that is heavily established in academia over the decades. Even the “definitions” are unscientific nonsense, and ever-changing. You would not accept such unscientific nonsense from me, but somehow you accept that filth from Stomebraker and his cult.

You can’t understand
__ server-level Locking
or
__ data-level Locking
while holding on to any shred of the MVCC fantasy that is heavily established in academia over the decades.

MVCC is anti-ACID, it will never work. The evidenced fact that every single Orable and PusGres “database” suffers massive problems due to the MVCC mindset is denied, an act that is enabled by repeating the totally fictitious *mantra* /Readers don't block writers, writers don't block readers/. the declaration is stupid, ridiculous, but hey, everyone is doing it; doing it; doing it.

(If you wish to get into this issue seriously, there is a separate thread open, pleaseengage over there.)

This thread is:
> It has to do with how to construct a Batch Transaction (MonthEnd Procedure at a bank or Move OldKey to NewKey).
But of course, we can’t help touching OLTP structures, because it is an OLTP environment, and that is the over-arching requirement.

-----------------------------
-- 3 Three Shells --
-----------------------------

See if you can step back and understand this.

A. Ten years ago, when you started interacting with me, your notion of a database, as programmed by academia, was:
__ the result of a collection of SQL command, and SQL is broken
__ literally, the physical only
__ ERD

B. As a consequence of that interaction, slowly, over time, it has progressed to:
__ the result of constraints, specified in SQL, and SQL may not be broken
__ introduction of some Logic
__ IDEF1X (ERD is anti-Relational)

C. And now recently:
__ the result of Predicates, specified in SQL, and SQL is not broken
__ introduction of essential Logic

Whereas [A] is one shell or core, with horrendous problems, [C] is an outer shell, that secures [A] in terms of data integrity, somewhat.

Likewise, as programmed by academia, you were stuck in
__ Suppress ACID
____ Erect Straw Men to make ACID and 1PL to look and smell bad
__ Elevate the fantasy of MVCC
____ thereby entirely denying the normal considerations for OLTP; ACID

We are just now starting to deal with a third shell [D], that further secures [A] “transactionally”, in terms of data integrity, completely.

This is totally new to academia, to you because you are the first and only academic trying to cross the chasm of ignorance of the industry, that the industry is, and has been since 1984 [as I know it]:

D.
__ OLTP Mindset (since 1960’s)
____ ACID Transactions only (since 1960’s, implement in all commercial SQL Platforms)
______ ACID in the server
______ ACID in every Transaction
____ Optimistic Locking
______ Optimistic Locking in each table
______ Optimistic Locking in every Transaction

You need to appreciate that as long as you harbour [2 Impediment] the Straw Men, there is no way that you can understand the [3 Three Shells] of protection, the third shell, the ACID Transaction context that has protected industry databases since the 1960’s.

Cheers
Derek

Nicola

> On Thursday, 17 June 2021 at 08:36:12 UTC+10, Derek Ignatius Asirvadem wrote:

--------------------------
-- 1 Correction --
--------------------------

> In this case, that means:
> -- UPDATE HotelChain.IsDeleted[ ABC ] = 1
> ---- WHERE HotelChain.IsDeleted[ ABC ] = 0
> or
> -- UPDATE HotelChain.IsLocked[ ABC ] = 1
> ---- WHERE HotelChain.IsLocked[ ABC ] = 0
>
> ...
>
> It also prevents:
> -- UPDATE Hotel.NumStar[]
> or
> -- UPDATE HotelRoomType.Description[]

Mistake, sorry, that should be:
It also prevents any Transaction containing:
-- UPDATE Hotel.NumStar[ ABC ]
or
-- UPDATE HotelRoomType.Description[ ABC ]
or
-- INSERT <AnyTableInHierarchy>[ ABC ]
or
-- DELETE <AnyTableInHierarchy>[ ABC ]

----------------------------
-- 2 Impediment --
----------------------------

> Yes, you do *not* have a correct understanding of Optimistic Locking, none of the academics do, they have only the Stonebraker MVCC Kool Aid, and they have been drinking it and circle-jerking while chanting the *Mantra* for decades. As I have tried to explain, Optimistic Locking is completely different to the Straw Man that academics erect as “optimistic locking”. This point (your question) has nothing to do with Optimistic Locking.

The term Optimistic Locking and the method (Standard) that it defines, is IBM [360] CICS/TCP from the early 1960’s. That is science. That is truth that does not change.

Any use of the term to mean anything other than that, is fraudulent; dishonest conduct; anti-science.

Which filth Stonebraker; Ellison; Date; Darwen; Fagin; etc, all engage in, in their suppression of the /RM/ and their promotion of RFS as “Relational”, their suppression of SQL and their promotion of non-sql as “SQL”.

You can’t understand
__ Optimistic Locking
(the OLTP requirement; two non-server deployments) while holding on to any shred of the “optimistic locking” Straw Man that is heavily established in academia over the decades.

You can’t understand
__ server-level Locking
or
__ data-level Locking
while holding on to any shred of the “2PL”; “S2PL”; “C2PL”; SS2PL”; etc, fantasies that is heavily established in academia over the decades. Even the “definitions” are unscientific nonsense, and ever-changing. You would not accept such unscientific nonsense from me, but somehow you accept that filth from Stomebraker and his cult.

You can’t understand
__ server-level Locking
or
__ data-level Locking
while holding on to any shred of the MVCC fantasy that is heavily established in academia over the decades.

MVCC is anti-ACID, it will never work. The evidenced fact that every single Orable and PusGres “database” suffers massive problems due to the MVCC mindset is denied, an act that is enabled by repeating the totally fictitious *mantra* /Readers don't block writers, writers don't block readers/. the declaration is stupid, ridiculous, but hey, everyone is doing it; doing it; doing it.

(If you wish to get into this issue seriously, there is a separate thread open, pleaseengage over there.)

This thread is:
> It has to do with how to construct a Batch Transaction (MonthEnd Procedure at a bank or Move OldKey to NewKey).
But of course, we can’t help touching OLTP structures, because it is an OLTP environment, and that is the over-arching requirement.

-----------------------------
-- 3 Three Shells --
-----------------------------

See if you can step back and understand this.

A. Ten years ago, when you started interacting with me, your notion of a database, as programmed by academia, was:
__ the result of a collection of SQL command, and SQL is broken
__ literally, the physical only
__ ERD

A.2 As a consequence of that interaction, slowly, over time, it has progressed to:
__ the result of constraints, specified in SQL, and SQL may not be broken
__ introduction of some Logic
__ IDEF1X (ERD is anti-Relational)

B. And now recently:
__ the result of Predicates, specified in SQL, and SQL is not broken
__ introduction of essential Logic

Whereas [A] is one shell or core, with horrendous problems, [B] is an outer shell, that secures [A] in terms of data integrity, somewhat.

Likewise, as programmed by academia, you were stuck in
__ Suppress ACID
____ Erect Straw Men to make ACID and 1PL to look and smell bad
__ Elevate the fantasy of MVCC
____ thereby entirely denying the normal considerations for OLTP; ACID

We are just now starting to deal with a third shell [C], that further secures [A] “transactionally”, in terms of data integrity, completely.

This is totally new to academia, to you because you are the first and only academic trying to cross the chasm of ignorance of the industry, that the industry is, and has been since 1984 [as I know it]:

C.
__ OLTP Mindset (since 1960’s)
____ ACID Transactions only (since 1960’s, implement in all commercial SQL Platforms)
______ ACID in the server
______ ACID in every Transaction
____ Optimistic Locking
______ Optimistic Locking in each table
______ Optimistic Locking in every Transaction

You need to appreciate that as long as you harbour [2 Impediment] the Straw Men for “ACID” instead of ACID, there is no way that you can understand the [3 Three Shells] of protection, the third shell, the ACID Transaction context that has protected industry databases since the 1960’s.

Cheers
Derek

On 2021-06-16, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> No, that is not correct.

As expected.

>>>>>
> You need some column that indicates a data-level lock. Standard
> columns include CreatedDtm; UpdatedDtm; etc, that can be used for this
> purpose [eg. set CreatedDtm to TODAY+1 or other fixed value [it is
> temporal, yes] ). You may have IsDeleted to support SoftDeletes.
> Let’s use the easy one, and say you have Soft Deletes. For clarity,
> you need a condition [ IsLocked ], and you implement that as indicated
> in one of the above columns, if not, you have a column [ IsLocked ].
>
> The fact that there is such a column, and that contention is
> understood and consciously minimised, *is* part of Optimistic Locking,
> part of the OLTP Standard.

That starts to sound vaguely familiar.

> But this is not an ACID Transaction, it is a Batch Transaction. Of
> course, we use the column that is there for OLTP purposes, for the now
> Batch purpose.
>
> In this case, that means:
> -- UPDATE HotelChain.IsDeleted[ ABC ] = 1
> ---- WHERE HotelChain.IsDeleted[ ABC ] = 0
> or
> -- UPDATE HotelChain.IsLocked[ ABC ] = 1
> ---- WHERE HotelChain.IsLocked[ ABC ] = 0
>
> That prevents both [a] and [b] in this:
>> another transaction may also attempt to
>> (a) update the record with HotelChain = ABC
>> (b) (and specifically update the HotelChain field)

Is it because each transaction checks UpdatedDtm and finds that it is
equal to TODAY+1, hence it immediately aborts?

>It also prevents any Transaction containing:
>-- UPDATE Hotel.NumStar[ ABC ]
>or
>-- UPDATE HotelRoomType.Description[ ABC ]
>or
>-- INSERT <AnyTableInHierarchy>[ ABC ]
>or
>-- DELETE <AnyTableInHierarchy>[ ABC ]

Such transactions perform the same check, right? And they abort
immediately, I assume, when they find that the top row has been marked
as deleted by the transaction that set UpdatedDtm. Is that correct?

> Separate point. That “another transaction” sounds suspiciously like
> another instance of the same transaction, using the same key. That
> doesn’t happen in the real world because the person who administers
> the first instance of changing ABC is one and the same person who
> administers the second instance of changing ABC. This sort of
> insanity is /also/ prevented by proper GUI design (a Tree with no
> circular references).

It may not happen under normal conditions. But I'd be very worried if
"it cannot happen" relies on correct (human or machine) behaviour.
A system should be resilient to improper and malicious behaviour as
well.

Nicola

Nicola

> On Thursday, 17 June 2021 at 19:27:16 UTC+10, Nicola wrote:
> > On 2021-06-16, Derek Ignatius Asirvadem wrote:
> > No, that is not correct.
> As expected.
> >>>>>
> > You need some column that indicates a data-level lock. Standard
> > columns include CreatedDtm; UpdatedDtm; etc, that can be used for this
> > purpose [eg. set CreatedDtm to TODAY+1 or other fixed value [it is
> > temporal, yes] ). You may have IsDeleted to support SoftDeletes.
> > Let’s use the easy one, and say you have Soft Deletes. For clarity,
> > you need a condition [ IsLocked ], and you implement that as indicated
> > in one of the above columns, if not, you have a column [ IsLocked ].
> >
> > The fact that there is such a column, and that contention is
> > understood and consciously minimised, *is* part of Optimistic Locking,
> > part of the OLTP Standard.
> That starts to sound vaguely familiar.
> > But this is not an ACID Transaction, it is a Batch Transaction. Of
> > course, we use the column that is there for OLTP purposes, for the now
> > Batch purpose.
> >
> > In this case, that means:
> > -- UPDATE HotelChain.IsDeleted[ ABC ] = 1
> > ---- WHERE HotelChain.IsDeleted[ ABC ] = 0
> > or
> > -- UPDATE HotelChain.IsLocked[ ABC ] = 1
> > ---- WHERE HotelChain.IsLocked[ ABC ] = 0
> >
> > That prevents both [a] and [b] in this:
> >> another transaction may also attempt to
> >> (a) update the record with HotelChain = ABC
> >> (b) (and specifically update the HotelChain field)
>
> Is it because each transaction checks UpdatedDtm and finds that it is
> equal to TODAY+1, hence it immediately aborts?

The /Today+1/ is for temporal tables, wherein a current SELECT does WHERE UpdatedDtm = MAX( UpdatedDtm) less-than-or-equal to today. It is just the standard method or construct to grab only the current row (exclude historic rows). (Whereas a temporal SELECT would grab the particular historic row..) I expect everyone who implements a temporal table [using Codd’s temporal definition, not the massive TwiddleDee & TwidleDumb monstrosity] to know that.

The /Today+1/ just makes the affected row (and the hierarchy below it) invisible to other users, a form of ISOLATION. Heck, we are on a planet named ACID, we are performing an [I]solation. Don’t let that confuse you. If the column is there, use it.

If IsDeleted is there, use it.

Else implement IsLocked, and use it.

Since God carved onto the OLTP stone tablet:
__ THOU SHALT NOT ATTEMPT ANYTHING THAT IS IMPOSSIBLE
yes, every Transaction (both ACID and this one) validates everything it will attempt, before attempting it, before the BEGIN TRAN. In our case, HotelChain is first. Because it is caught in the VALIDATE block, before the BEGIN TRAN, there is no “abort” or ROLLBACK TRAN.

The corollary to that Commandment is of course:
__ Thous Shalt Not Begin Anything That Will Fail, That Thou Canst Not Complete.

I am scared of fire and brimstone. The CICS/TCP guys taught me this when I was still loading COBOL programs into a mainframe, in the form of punch-card decks.

What you guys call “2 phase” is anti-ACID, it starts a transaction and then fiddles and farts while rambling, meaning time spans between requests. As per the wiki cesspool article, in the “SS2PL” that we have, there is only one “phase”. But we don’t start that “phase” until the previous “phase” of VALIDATE completes successfully. And we have no time spans in-between operations.

I expect that you understand the following:
__ in order to comply with [A]tomic, the BEGIN and COMMIT/ROLLBACK must be in one code block in a stored proc, never in the client
__ the caller of the Transaction must execute the exact same VALIDATE block, before calling the Transaction

--------

> >It also prevents any Transaction containing:
> >-- UPDATE Hotel.NumStar[ ABC ]
> >or
> >-- UPDATE HotelRoomType.Description[ ABC ]
> >or
> >-- INSERT <AnyTableInHierarchy>[ ABC ]
> >or
> >-- DELETE <AnyTableInHierarchy>[ ABC ]
>
> Such transactions perform the same check, right? And they abort
> immediately, I assume, when they find that the top row has been marked
> as deleted by the transaction that set UpdatedDtm. Is that correct?

Yes.
Every Transaction that touches the HotelChain hierarchy, any level.
As detailed above, not “abort”, but never start.

Not necessarily “deleted” (which depends on the column that is available to be used for this purpose), but ISOLATED for the duration of the Transaction. When it has completed, the entire NewKey hierarchy is available.

> > Separate point. That “another transaction” sounds suspiciously like
> > another instance of the same transaction, using the same key. That
> > doesn’t happen in the real world because the person who administers
> > the first instance of changing ABC is one and the same person who
> > administers the second instance of changing ABC. This sort of
> > insanity is /also/ prevented by proper GUI design (a Tree with no
> > circular references).
>
> It may not happen under normal conditions. But I'd be very worried if
> "it cannot happen" relies on correct (human or machine) behaviour.
> A system should be resilient to improper and malicious behaviour as
> well.

Absolutely. I did not say “it cannot happen”. It is prevented from happening, by the Transaction standard. That was an explanation, that it is actually prevented at a higher level (in any proper GUI), before the Transaction is called. Therefore “/also/“.

Cheers
Derek

On 2021-06-17, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> The /Today+1/ is for temporal tables, wherein a current SELECT does
> WHERE UpdatedDtm = MAX( UpdatedDtm) less-than-or-equal to today. It
> is just the standard method or construct to grab only the current row
> (exclude historic rows). (Whereas a temporal SELECT would grab the
> particular historic row.) I expect everyone who implements a temporal
> table [using Codd’s temporal definition, not the massive TwiddleDee
> & TwidleDumb monstrosity] to know that.
>
> The /Today+1/ just makes the affected row (and the hierarchy below
> it) invisible to other users, a form of ISOLATION. Heck, we are on
> a planet named ACID, we are performing an [I]solation. Don’t let that
> confuse you. If the column is there, use it.
>
> If IsDeleted is there, use it.
>
> Else implement IsLocked, and use it.
>
> Since God carved onto the OLTP stone tablet:
> __ THOU SHALT NOT ATTEMPT ANYTHING THAT IS IMPOSSIBLE
> yes, every Transaction (both ACID and this one) validates everything
> it will attempt, before attempting it, before the BEGIN TRAN. In our
> case, HotelChain is first. Because it is caught in the VALIDATE
> block, before the BEGIN TRAN, there is no “abort” or ROLLBACK TRAN.

Wait, what is a VALIDATE block? If an atomic processing unit is
delimited by BEGIN TRAN...COMMIT, how can something before BEGIN TRAN be
considered part of that unit? If it is not, then how do you ensure that
nothing happens between the instant VALIDATE completes and the instant
BEGIN TRAN is executed?

And I have a couple more questions, if you don't mind:

1. What you are describing is built on top of the standard locking
mechanism provided by the DBMS. If I understand correctly, it requires
the transactions to be coded to perform explicit checks before accessing
the data. At the outset, that sounds like something that can become very
complicated and error-prone. E.g., it seems that a transaction to insert
a new ReadingMeasure should check many things (HotelChain, CountryCode,
...., Application) before proceeding.

Or, wait, such a transaction would be coded in a "simple" way (insert
into Reading, insert into ReadingMeasure) and then two things may
happen: it is either executed before the batch transaction has "reached"
Reading, in which case it would succeed (and then it would be updated by
the batch transaction); or it is executed after the batch transaction
has updated Reading, hence inserting the new reading would raise
a foreign key violation (and it should be retried with the new key).

Or something else?

2. You are describing an OLTP context, but you have claimed several
times that the same database can serve both OLTP and OLAP workloads. Do
you code analytic queries along the same principles to avoid
locking too many records?

> The corollary to that Commandment is of course:
> __ Thous Shalt Not Begin Anything That Will Fail, That Thou Canst Not
> Complete.

That is worth the "optimistic" label!

> What you guys call “2 phase” is anti-ACID, it starts a transaction and
> then fiddles and farts while rambling, meaning time spans between
> requests. As per the wiki cesspool article, in the “SS2PL” that we
> have, there is only one “phase”.

Ok.

> But we don’t start that “phase” until the previous “phase” of VALIDATE
> completes successfully. And we have no time spans in-between
> operations.

See above. Not clear how you'd achieve that.

Nicola

Nicola

Please consider in the truth of learning what an OLTP context is, in a Ordinary Locking server (falsely known as “1PL’; “2PL”; “C2PL” ... “SS2PL”; etc). Not in the MVCC mindset, and definitely not what you think ACID or ACID Transaction is. But what it really is, in the science, in the platforms since 1960.. SQL Platforms sice 1984. That excludes Stonebraker; Ingres; its zombie son PusGres; and Oracle.

Second, please consider the OLTP context, all ACDI Transactions sans the Batch Transaction, only. After that is clear (no questions) then add the Batch Transaction (high-end OLTP only) to the consideration.

> On Saturday, 19 June 2021 at 00:22:26 UTC+10, Nicola wrote:
> > On 2021-06-17, Derek Ignatius Asirvadem wrote:
> >
> > The /Today+1/ is for temporal tables, wherein a current SELECT does
> > WHERE UpdatedDtm = MAX( UpdatedDtm) less-than-or-equal to today. It
> > is just the standard method or construct to grab only the current row
> > (exclude historic rows). (Whereas a temporal SELECT would grab the
> > particular historic row.) I expect everyone who implements a temporal
> > table [using Codd’s temporal definition, not the massive TwiddleDee
> > & TwidleDumb monstrosity] to know that.
> >
> > The /Today+1/ just makes the affected row (and the hierarchy below
> > it) invisible to other users, a form of ISOLATION. Heck, we are on
> > a planet named ACID, we are performing an [I]solation. Don’t let that
> > confuse you. If the column is there, use it.
> >
> > If IsDeleted is there, use it.
> >
> > Else implement IsLocked, and use it.
> >
> > Since God carved onto the OLTP stone tablet:

> > __ THOU SHALT NOT ATTEMPT ANYTHING THAT IS IMPOSSIBLE

> > yes, every Transaction (both ACID and this one) validates everything
> > it will attempt, before attempting it, before the BEGIN TRAN. In our
> > case, HotelChain is first. Because it is caught in the VALIDATE
> > block, before the BEGIN TRAN, there is no “abort” or ROLLBACK TRAN.
>
> Wait, what is a VALIDATE block? If an atomic processing unit is
> delimited by BEGIN TRAN...COMMIT, how can something before BEGIN TRAN be
> considered part of that unit? If it is not, then how do you ensure that
> nothing happens between the instant VALIDATE completes and the instant
> BEGIN TRAN is executed?

All code must be standard, eg. there may be a template. The Transaction stored proc (as distinct from the ACID Transaction in it, that it is named for) contains at least three code blocks, four in high-end OLTP. The EXECUTE Block is Atomic in the ACID sense. The stored proc is Atomic in the sense that it is a single code segment that the caller calls and is executed (partly or wholly).

(In the simple sense, for newbies, ACID Transaction = stored proc. But of course, *all* code is standardised, we have templates, the stored proc is not ONLY the ACID Transaction, it has to have a structure; a Form; error handling; etc.)

______________________________________________
0. THERE IS ONLY ONE VERSION OF ANY FACT

__ Schizophrenics are prevented from writing code of any kind.

__________________
1. VALIDATE Block
__ In CICS/TCP/COBOL terms, this is a PREPARE block.
You know the Fives P’s, absolutely essential for succeeding at anything ? Eg. an Olympic swimmer; a deer hunter; a programmer of ACID Transactions.
__ Proper Preparation Prevents Poor Performance.

>>>>
We know we are locking rows, implicitly, by virtue of the SQL commands we issue, but it is the server that does the locking and the resolution.
• We know that Reads are locked for the duration of the Read operation, and Writes are locked until the COMMIT/ROLLBACK.
__• We can also request a Read to be locked until COMMIT/ROLLBACK, but that requires a Transaction to be opened, and a server that [is genuine OLTP and therefore] allows it.
<<<<

Code:
• Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• RETURN on any failure.
• Check NOT_EXISTS all rows that must not exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• RETURN on any failure.
• Transaction NOT yet started
• All locking is transient and very short duration
• TransactionLog is not touched
• Fall into ...

__________________
2. EXECUTE Block
* BEGIN TRAN
• TransactionLog activity commenced for this xact
• Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• ROLLBACK & RETURN on any failure.
__• the only relevant locking starts here
__• if blocked by another xact, you are blocked here (NOT after the first verb below)
__• the longest wait is for the longest xact in the system that touches the page

So here we code each SELECT as:
__ IF EXISTS (
____ SELECT ...
______ FROM Hotel
______ WHERE ...
______ { FOR UPDATE | HOLDLOCK } -- platform-specific syntax, “beyond” SQL
____ )

__• <-[2.1]

• Check NOT_EXISTS all rows that must not exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• ROLLBACK & RETURN on any failure.
• (INSERT/UPDATE/DELETE)
__• ROLLBACK & RETURN on any failure.

• COMMIT TRAN
• TransactionLog is released.

___________________
3. CLEAN-UP Block
• All error handling
• ROLLBACK TRAN (if structured, the only ROLLBACK)
• <-[3.1]
• RETURN (if structured, the only RETURN)

______
Client
There is ordinary interplay between the Client and the Server. The client may now be a GUI or a webpage or middleware in an app server.

The VALIDATE block [1] is repeated in the client. This is not a duplicate, because it is complying with the Commandment, it would be stupid to attempt something that will fail. We do not need to engage server lock resources and the TransactionLog to find out that something will fail, we can check for it without engaging that. Besides, such an act would get the pages into the cache, if it is not already there, and then, when the EXECUTE block starts, the required pages are in the cache, warm and buttered, waiting to be served up.
_________________
The ACID Transaction is bounded by BEGIN TRAN...COMMIT/ROLLBACK TRAN. It doesn’t even start unless it can complete, which is validated before the start.

> how do you ensure that
> nothing happens between the instant VALIDATE completes and the instant
> BEGIN TRAN is executed?

That duration between instants is:
a. not relevant [assuming you understand the above], and
b. microseconds, because it is between two consecutive steps inside a contiguous stored proc, which is running as object + plan code (compiled and a full resource plan established), and
c. harmless because no resources are held.

__________________
> If an atomic processing unit is
> delimited by BEGIN TRAN...COMMIT,

The /processing unit/ wrt ACID Transaction which indeed must be Atomic, is bounded by BEGIN-COMMIT TRAN, in one contiguous code block.

The /processing unit/ that contains it is not Atomic in the ACID sense, but it is Atomic by design, a contiguous code block; and optimised (compiled + resource plan). Eg. I do not allow that to be non-contiguous. The two need not be the same /processing unit/.

This is also why the BEGIN TRAN must never be in the client.
__________________
Batch Transaction

Add:

2.1 Add the data-level lock
___ BEGIN TRAN
___ UPDATE HotelChainIsDeleted[ @HotelChain ] = 1
___ COMMIT TRAN

3.1 Remove the data-level lock (in our example this ois not required, because it is the last row deleted).
___ BEGIN TRAN
___ UPDATE HotelChainIsDeleted[ @HotelChain ] = 0
___ COMMIT TRAN

You may consider the BEGIN-COMMIT TRAN superfluous. No, They are demanded by the standard. Further, they are anchors for code investigation, and on the server vis-a-vis monitoring metrics.
_________________

> And I have a couple more questions, if you don't mind:

I would be pleased to answer.

> 1. What you are describing is built on top of the standard locking
> mechanism provided by the DBMS.

If you mean the normal ACID Transaction in the full OLTP Context, no. It is aware of, and implicitly uses the locking in the server (whatever that is).

If you mean the Batch Transaction, yes. I said so at the start. That is why I call it a data-level lock, it has to be done in data, we have no direct access to the locks, and we never do idiotic things such as:
__ LOCK TABLE

> If I understand correctly, it requires
> the transactions to be coded to perform explicit checks before accessing
> the data.

Yes.
All ACID Transactions.
It is the same “explicit” checks as the “explicit” operations it will attempt, no less, no more.
We have been doing that since 1960, on all OLTP systems.
Welcome to the industry in the real world, that you guys have never served.
Thank you for visiting.
Please come again.

> At the outset, that sounds like something that can become very
> complicated and error-prone.

How ? Why ?
Sure, we use templates, and the developer has to be educated in more than the schizophrenic MVCC mindset, but I can’t see how it is “complicated” (see detail above, or “error-prone”. In any case, the errors will be caught in development DEV, long before TEST and UAT.

Nicola

> On Saturday, 19 June 2021 at 17:20:51 UTC+10, Derek Ignatius Asirvadem wrote:

> Read locks

The previous post is a bit long, and that item was addressed in a few places. In case it isn't clear:
__ Read locks outside a Transaction are transient, held for the duration of the read operation
__ Read locks inside a Transaction are held until COMMIT/ROLLBACK
____ that is the SQL requirement, to support ACID

("MVCC" systems do none of that ... and try to desperately clean up a mountain of false versions of rows (not even pages!), that are distributed across the entire file space, when one person hits the [Save] button. And contrary to the cult hymn, it produces conditions that cannot be resolved, and resorts to "2PL". And yet still produces failures akin to deadlock. But the cult love their denial, and just sing the hymn louder.)

As stated in:
____ https://www.softwaregems.com.au/Documents/Article/Sybase%20Lock%20Manager/Sybase%20Lock%20Manager.pdf

Cheers
Derek

Nicola

> On Saturday, 19 June 2021 at 17:52:18 UTC+10, Derek Ignatius Asirvadem wrote:

> > On Saturday, 19 June 2021 at 17:20:51 UTC+10, Derek Ignatius Asirvadem wrote:

Sorry, stuffed up yet again. It has been at least 12 years since the last time I explained this.

> > Read locks

> __ Read locks outside a Transaction are transient, held for the duration of the read operation

No. This:
__ Read locks outside a Transaction are transient, held for the duration of the Statement
Of course, that is for reports. That is so that the entire ResultSet or SELECT set is not affected by Transactions. Yet another ACID SQL requirement, that MVCC doesn't even try to achieve. Oracle has made some improvements over the decades, which they falsely say is fixed, but it is still not compliant.

Cheers
Derek

On 2021-06-19, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> __________________
> 1. VALIDATE Block
> Code:
> • Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> __• RETURN on any failure.
> • Check NOT_EXISTS all rows that must not exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> __• RETURN on any failure.
> • Transaction NOT yet started
> • All locking is transient and very short duration
> • TransactionLog is not touched
> • Fall into ...

Ok.

> __________________
> 2. EXECUTE Block
> * BEGIN TRAN
> • TransactionLog activity commenced for this xact
> • Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> __• ROLLBACK & RETURN on any failure.
> __• the only relevant locking starts here
> __• if blocked by another xact, you are blocked here (NOT after the first verb below)
> __• the longest wait is for the longest xact in the system that touches the page
>
> So here we code each SELECT as:
> __ IF EXISTS (
> ____ SELECT ...
> ______ FROM Hotel
> ______ WHERE ...
> ______ { FOR UPDATE | HOLDLOCK } -- platform-specific syntax, “beyond” SQL
> ____ )
>
> __• <-[2.1]
>
> • Check NOT_EXISTS all rows that must not exist, in order for the
> intended rows (INSERT/UPDATE/DELETE) to succeed.
> __• ROLLBACK & RETURN on any failure.
> • (INSERT/UPDATE/DELETE)
> __• ROLLBACK & RETURN on any failure.
>
> • COMMIT TRAN
> • TransactionLog is released.

Ok.

> ___________________
> 3. CLEAN-UP Block
> • All error handling
> • ROLLBACK TRAN (if structured, the only ROLLBACK)
> • <-[3.1]
> • RETURN (if structured, the only RETURN)

Ok.

> ______
> Client
> There is ordinary interplay between the Client and the Server. The
> client may now be a GUI or a webpage or middleware in an app server.
>
> The VALIDATE block [1] is repeated in the client.

I assume that this means validating data without interacting with the
database, e.g., to check that the input data is within valid ranges,
etc.

>> how do you ensure that
>> nothing happens between the instant VALIDATE completes and the instant
>> BEGIN TRAN is executed?
>
> That duration between instants is:
> a. not relevant [assuming you understand the above], and
> b. microseconds, because it is between two consecutive steps inside
> a contiguous stored proc, which is running as object + plan code
> (compiled and a full resource plan established), and

That guarantees that interference is unlikely, but not impossible.
Otherwise, you would not need this:

> * BEGIN TRAN
> • TransactionLog activity commenced for this xact
> • Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> __• ROLLBACK & RETURN on any failure.

> c. harmless because no resources are held.

That is an interesting approach. It is not obvious, IMO, that validating
the data the way you suggest is advantageous (I am not saying that it is
not, mind you!). Whether it is or not, it seems to depend on a number of
factors, such as number of concurrent transactions, duration of
transactions, number of touched resources, etc.

Validation introduces an overhead, but it prevents transactions to start
and acquire locks unless there is a very good chance to acquire them
all, so overall it may be a win.

To summarize, you could implement transactions in at least three ways:

(1) just BEGIN TRAN; INSERT/UPDATE/DELETE; END TRAN:
(2) BEGIN TRAN; SELECT FOR UPDATE; INSERT/UPDATE/DELETE; END TRAN;
(3) VALIDATE, then (2).

And in your experience (3) is the best approach.

> Batch Transaction
>
> Add:
>
> 2.1 Add the data-level lock
> ___ BEGIN TRAN
> ___ UPDATE HotelChainIsDeleted[ @HotelChain ] = 1
> ___ COMMIT TRAN

Ok.

> 3.1 Remove the data-level lock (in our example this ois not required, because it is the last row deleted).
> ___ BEGIN TRAN
> ___ UPDATE HotelChainIsDeleted[ @HotelChain ] = 0
> ___ COMMIT TRAN

Ok.

> You may consider the BEGIN-COMMIT TRAN superfluous. No, They are
> demanded by the standard. Further, they are anchors for code
> investigation, and on the server vis-a-vis monitoring metrics.

Ok.

>> 1. What you are describing is built on top of the standard locking
>> mechanism provided by the DBMS.
>
> If you mean the normal ACID Transaction in the full OLTP Context, no.
> It is aware of, and implicitly uses the locking in the server
> (whatever that is).
>
> If you mean the Batch Transaction, yes. I said so at the start. That
> is why I call it a data-level lock, it has to be done in data, we have
> no direct access to the locks, and we never do idiotic things such as:
> __ LOCK TABLE

Ok, that is all pretty clear now.

>> OLAP Reports
> Why do you worry about locks in OLAP reports ?
> Who cares.
> Read locks are extremely short, and transient (held for the duration
> of the read operation, not the Transaction). We can’t control Read
> locks, even implicitly

Are you saying that OLAP queries run in a lower isolation level (say,
READ COMMITTED)?

> But still, that is not Optimistic Locking. And in this post, there is
> nothing about Optimistic Locking.

Agreed.

Thanks,
Nicola

Nicola

> On Tuesday, 22 June 2021 at 20:35:05 UTC+10, Nicola wrote:
> > On 2021-06-19, Derek Ignatius Asirvadem wrote:

> > ______
> > Client
> > There is ordinary interplay between the Client and the Server. The
> > client may now be a GUI or a webpage or middleware in an app server.
> >
> > The VALIDATE block [1] is repeated in the client.
> I assume that this means validating data without interacting with the
> database, e.g., to check that the input data is within valid ranges,
> etc.

No.
Any GUI or input system has to do those checks on input fields, that is fundamental, not usually mentioned.

Let’s say there are 20 WIDGETS currently in the inventory:
__ why would you allow an user to purchase 6 WIJJETS [that do not exist] ???
__ Why would you allow an user to purchase 100 WIDGETS [that are not in stock] ???

Why would you let users enter any old nonsense all the way through the GUI; then at the end try it all against the database (the first failure would return, the second and subs would not even get checked); then find out that it failed; then try the whole rigmarole again ???

Why would you NOT check for validity at each and every field (as the user progresses through the fields) ??? Did you not genuflect to the Commandment ? Why is the Client (GUI or whatever) excluded from being part of the system ?

By:
> > The VALIDATE block [1] is repeated in the client.
I mean the VALIDATE block [1] is repeated in the client. All of this:

>> [Modified Slightly for the GUI Context]
__________________
1. VALIDATE Block
__ Proper Preparation Prevents Poor Performance.

>>>>
We know we are locking rows, implicitly, by virtue of the SQL commands we issue, but it is the server that does the locking and the resolution.
• We know that Reads are locked for the duration of the Statement, and Writes are locked until the COMMIT/ROLLBACK.
__• We can also request a Read to be locked until COMMIT/ROLLBACK, but that requires a Transaction to be opened, and a server that [is genuine OLTP and therefore] allows it.
<<<<

Code:
• Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• On any failure, error message; clear field; allow re-entry.
• Check NOT_EXISTS all rows that must not exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• On any failure, error message; clear field; allow re-entry.
• Transaction NOT yet started
• All locking is transient and very short duration
• TransactionLog is not touched
<<

When complete (eg. hitting the <Save> button), execute the Transaction stored proc.

Therefore, the Xact call when issued at the end of the GUI interaction has been VALIDATED before the attempt. While the user was fiddling around (between the presentation of the GUI dialogue and the instant the <Save> button was pressed, no rows are locked; no contention is inflicted on other users, due to invalid attempts or worse, transactions partly executed before being rolled back.

If that is NOT done in the Client, the user is encumbered with entering any old nonsense; attempting it against the database (via the Transaction only); finding out that something or other failed (one at a time); and retrying; retrying; retrying. The notion is ridiculous.

> >> how do you ensure that
> >> nothing happens between the instant VALIDATE completes and the instant
> >> BEGIN TRAN is executed?
> >
> > That duration between instants is:
> > a. not relevant [assuming you understand the above], and
> > b. microseconds, because it is between two consecutive steps inside
> > a contiguous stored proc, which is running as object + plan code
> > (compiled and a full resource plan established), and
>
> That guarantees that interference is unlikely, but not impossible.

(Well, if it “guarantees that interference is unlikely” then it is impossible, if it is “not impossible” then it is not a guarantee.)
So what. The code is interrupted at that point. It is a normal possibility, catered for: if the code segment is interrupted at that or any other point, the Transaction is rolled back and an error status is returned to the caller. At that particular point, no rows locked; Transaction not started. If at a later point, Transaction started and partially complete, then rolled back. So what. Ordinary fare. Why is that point of possible failure more important than any other point, why do we need to “ensure nothing happens between end-VALIDATE and BEGIN-TRAN ?

> Otherwise, you would not need this:
> > * BEGIN TRAN
> > • TransactionLog activity commenced for this xact
> > • Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> > __• ROLLBACK & RETURN on any failure.

Huh ?
I need that because I love God and His Commandments; because I do not attempt actions that will fail; I am sociable and do not inflict resource-locking on other users unless I really have to. I need to separate the VALIDATE & EXECUTE blocks, because the VALIDATE blocks prepares but does not change anything, and the EXECUTE block performs the changes in a structured Transaction, only if the VALIDATE block succeeds.

OTOH, if you are concerned about data in the database changing between the point in the VALIDATE block that checks a particular row, and the point the EXECUTE block that updates the relevant row, then yes, that is not catered for in the above description. However, that [and one other condition) is covered by Optimistic Locking.

> > c. harmless because no resources are held.
> That is an interesting approach. It is not obvious, IMO, that validating
> the data the way you suggest is advantageous (I am not saying that it is
> not, mind you!).

And there is a VALIDATION being done in the Client, the duration of which cannot be controlled due to user interaction.

It is not merely advantageous, which of course it is, it is demanded by the OLTP Standard.

> Whether it is or not, it seems to depend on a number of
> factors, such as number of concurrent transactions, duration of
> transactions, number of touched resources, etc.

???
It is advantageous is all circumstances. The more active and contentious the system is, the more advantageous the Standard is.

> Validation introduces an overhead, but it prevents transactions to start
> and acquire locks unless there is a very good chance to acquire them
> all, so overall it may be a win.

Yes.
I would not call it an “overhead”.
Checking the depth of the pond before diving in, or checking that a Part exists before ordering it, is not “overhead”, it is a demanded part of the task.

> To summarize, you could implement transactions in at least three ways:
>
> (1) just BEGIN TRAN; INSERT/UPDATE/DELETE; END TRAN:
> (2) BEGIN TRAN; SELECT FOR UPDATE; INSERT/UPDATE/DELETE; END TRAN;
> (3) VALIDATE, then (2).
>
> And in your experience (3) is the best approach.

Can you suggest any other method, that locks the smallest number of resources, AND locks said resources for the shortest duration, AND only when those resources are sure to be touched ?

No, not merely my experience.
a. the way it was done, and the way it is still done, on “legacy” mainframe systems.
b. the way it is done on genuine OLTP systems (otherwise they have no right to the /OLTP/ label). These are well-known and published methods. Even for the totally ignorant, upon using a genuine OLTP server, they would read it in the manuals, the same way that the insanity of MVCC is discussed in the PusGres manuals, but with honesty not fantasy. Ie. specific methods for reducing contention within the particular product are given.

[1][2] would be a very stupid app, that allows the user keep banging away at things that do not exist.

[3] is not best. And it certainly fails the OLTP requirement. You need:
4. VALIDATE all intended changes IN THE CLIENT during user interaction
__ upon <Save> execute the Transaction sp
____ which performs [3]
______ which performs [2]

________

> >> OLAP Reports
> > Why do you worry about locks in OLAP reports ?
> > Who cares.
> > Read locks are extremely short, and transient (held for the duration
> > of the read operation [Statement], not the Transaction). We can’t control Read
> > locks, even implicitly
>
> Are you saying that OLAP queries run in a lower isolation level (say,
> READ COMMITTED)?

Of course. That is what that ISOLATION LEVEL[ READ COMMITTED ] is there for.

Not just OLAP queries, but all SELECTs outside a Transaction. That [ISOLATION LEVEL] is an ANSI SQL requirement, within the ACID requirement. We exclude [wait for completion of] uncommitted changes, but we do not need SERIALISED, in fact we want massively parallel, and we want the whole result set to be integral within itself (all internal dependencies intact and true).

Did you not read the link I gave for the Sybase Lock Manager ? Page 2, bottom left, /Resolution/ table. I do not refer you to some doc unless it is directly related to the item at issue in the post.

In the Sybase world, for decades, we do not bother with REPEATABLE READ, it defaults to SERIALISABLE, which although it is a higher level of ISOLATION, works perfectly. In 2000, with the introduction of DRL/DPL data structures to provide improved speed for RFS (SAP is RFS), yes, they implemented REPEATABLE READ.

Nicola

> On Wednesday, 23 June 2021 at 09:52:25 UTC+10, Derek Ignatius Asirvadem wrote:
> > On Tuesday, 22 June 2021 at 20:35:05 UTC+10, Nicola wrote:
> > > On 2021-06-19, Derek Ignatius Asirvadem wrote:
>
> > > ______
> > > Client
> > > There is ordinary interplay between the Client and the Server. The
> > > client may now be a GUI or a webpage or middleware in an app server.
> > >
> > > The VALIDATE block [1] is repeated in the client.
> >
> > I assume that this means validating data without interacting with the
> > database, e.g., to check that the input data is within valid ranges,
> > etc.
>
> No.
> Any GUI or input system has to do those checks on input fields, that is fundamental, not usually mentioned.
>
> Let’s say there are 20 WIDGETS currently in the inventory:
> __ why would you allow an user to purchase 6 WIJJETS [that do not exist] ???
> __ Why would you allow an user to purchase 100 WIDGETS [that are not in stock] ???

What does normal GUI interaction mean to you ? Why does it exclude database interaction ? Why would you NOT restrict the user to choosing only those Parts that do exist in the database, and to a purchase quantity that is currently in stock ? Otherwise the GUI is insane, the user interaction is insane.

Cheers
Derek

Nicola

> On Tuesday, 22 June 2021 at 20:35:05 UTC+10, Nicola wrote:
> > On 2021-06-19, Derek Ignatius Asirvadem wrote:

> > ______
> > Client
> > There is ordinary interplay between the Client and the Server. The
> > client may now be a GUI or a webpage or middleware in an app server.
> >
> > The VALIDATE block [1] is repeated in the client.
> I assume that this means validating data without interacting with the
> database, e.g., to check that the input data is within valid ranges,
> etc.

No.
Any GUI or input system has to do those checks on input fields, that is fundamental, not usually mentioned.

Let’s say there are 20 WIDGETS currently in the inventory:
__ why would you allow an user to purchase 6 WIJJETS [that do not exist] ???
__ Why would you allow an user to purchase 100 WIDGETS [that are not in stock] ???

What does normal GUI interaction mean to you ? Why does it exclude database interaction ? Why would you NOT restrict the user to choosing only those Parts that do exist in the database, and to a purchase quantity that is currently in stock ? Otherwise the GUI is insane, the user interaction is insane.

For decades, GUIs have had Drop-Down fields, which are obtained from the db upon launching the dialogue. And fields that are searched-and-filled as the user types a few characters, which are obtained from the db.

Why would you design a GUI that does NOT interact with the db in order to validate its intent ... before it executes its intent ???

Why would you let users enter any old nonsense all the way through the GUI; then at the end try it all against the database (the first failure would return, the second and subs would not even get checked); then find out that it failed; then try the whole rigmarole again ???

Why would you NOT check for validity at each and every field (as the user progresses through the fields) ??? Did you not genuflect to the Commandment ? Why is the Client (GUI or whatever) excluded from being part of the OLTP system, and therefore subject to OLTP Standards ?

Why would you allow an user to even dream about a Part that you do not have in the inventory [WIJJET] ?

By:
> > The VALIDATE block [1] is repeated in the client.
I mean /the VALIDATE block [1] is repeated in the client/. All of this, (modified Slightly for the GUI Context):

>>
__________________
1. VALIDATE Block
__ Proper Preparation Prevents Poor Performance.

>>>>
We know we are locking rows, implicitly, by virtue of the SQL commands we issue, but it is the server that does the locking and the resolution.
• We know that Reads are locked for the duration of the Statement, Read Locks are transient.
• This is interactive, thus duration cannot be controlled, thus no locks held
<<<<

Code:
• Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• On any failure, error message; clear field; allow re-entry.
• Check NOT_EXISTS all rows that must not exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
__• On any failure, error message; clear field; allow re-entry.
• Transaction NOT yet started
• No locks held
• TransactionLog is not touched
<<

When complete (eg. hitting the <Save> button), execute the Transaction stored proc.

Therefore, the Xact call when issued at the end of the GUI interaction has been VALIDATED before the attempt. While the user was fiddling around (between the presentation of the GUI dialogue and the instant the <Save> button was pressed, no rows are locked; no contention is inflicted on other users, due to invalid attempts or worse, transactions partly executed before being rolled back.

If that is NOT done in the Client, the user is encumbered with entering any old nonsense; attempting it against the database (via the Transaction only); finding out that something or other failed (one at a time); and retrying; retrying; retrying. The notion is ridiculous.

> >> how do you ensure that
> >> nothing happens between the instant VALIDATE completes and the instant
> >> BEGIN TRAN is executed?
> >
> > That duration between instants is:
> > a. not relevant [assuming you understand the above], and
> > b. microseconds, because it is between two consecutive steps inside
> > a contiguous stored proc, which is running as object + plan code
> > (compiled and a full resource plan established), and
>
> That guarantees that interference is unlikely, but not impossible.

(Well, if it “guarantees that interference is unlikely” then it is impossible, if it is “not impossible” then it is not a guarantee.)
So what. The code is interrupted at that point. It is a normal possibility, catered for: if the code segment is interrupted at that or any other point, the Transaction is rolled back and an error status is returned to the caller. At that particular point, no rows locked; Transaction not started. If at a later point, Transaction started and partially complete, then rolled back. So what. Ordinary fare. Why is that point of possible failure more important than any other point, why do we need to “ensure nothing happens between end-VALIDATE and BEGIN-TRAN ?

> Otherwise, you would not need this:
> > * BEGIN TRAN
> > • TransactionLog activity commenced for this xact
> > • Read all rows that must exist, in order for the intended rows (INSERT/UPDATE/DELETE) to succeed.
> > __• ROLLBACK & RETURN on any failure.

Huh ?
I need that because I love God and His Commandments; because I do not attempt actions that will fail; I am sociable and do not inflict resource-locking on other users unless I really have to. I need to separate the VALIDATE & EXECUTE blocks, because the VALIDATE blocks prepares but does not change anything, and the EXECUTE block performs the changes in a structured Transaction, only if the VALIDATE block succeeds.

OTOH, if you are concerned about data in the database changing between the point in the VALIDATE block that checks a particular row, and the point the EXECUTE block that updates the same particular row, then yes, that is not catered for in the above description. However, that [and one other major problem, that can be eliminated) is covered by Optimistic Locking.

> > c. harmless because no resources are held.
> That is an interesting approach. It is not obvious, IMO, that validating
> the data the way you suggest is advantageous (I am not saying that it is
> not, mind you!).

And there is a VALIDATION being done in the Client, the duration of which cannot be controlled due to user interaction.

It is not merely advantageous, which of course it is, it is demanded by the OLTP Standard.

> Whether it is or not, it seems to depend on a number of
> factors, such as number of concurrent transactions, duration of
> transactions, number of touched resources, etc.

Is not every request to the server “dependent” on those same factors ??? DO you have a method of calling the server such that those loads do NOT affect the Transaction ??? Maybe the MVCC fantasy of a private version of the entire database. No, this is real life.

The duration of transactions is minimised, to the absolute minimu, by the OLTP Standard. In case it is not clear, unlike the MVCC asylum:
a. we do not hold locks outside a Transaction, and
b. we do not BEGIN TRAN except to execute a Transaction, and
c. we do not allow user interaction after a BEGIN TRAN.

It is advantageous is all circumstances. The more active and contentious the system is, the more advantageous the OLTP Standard is.

> Validation introduces an overhead, but it prevents transactions to start
> and acquire locks unless there is a very good chance to acquire them
> all, so overall it may be a win.

Yes.
I would not call it an “overhead”.
Checking the depth of the pond before diving in, or checking that a Part exists before ordering it, is not “overhead”, it is a demanded part of the task.

> To summarize, you could implement transactions in at least three ways:
>
> (1) just BEGIN TRAN; INSERT/UPDATE/DELETE; END TRAN:
> (2) BEGIN TRAN; SELECT FOR UPDATE; INSERT/UPDATE/DELETE; END TRAN;
> (3) VALIDATE, then (2).
>
> And in your experience (3) is the best approach.

Can you suggest any other method, that locks the smallest number of resources, AND locks said resources for the shortest duration, AND only when those resources are sure to be touched ?

No, not merely my experience.
a. the way it was done, and the way it is still done, on “legacy” mainframe systems.
b. the way it is done on genuine OLTP systems (otherwise they have no right to the /OLTP/ label). These are well-known and published methods. Even for the totally ignorant, upon using a genuine OLTP server, they would read it in the manuals, the same way that the insanity of MVCC is discussed in the PusGres manuals, but with honesty not fantasy. Ie. specific methods for reducing contention within the particular product are given.

As stated severally, about important matters (eg. God; truth) or about science (real science not the Date;Darwen; Fagin; etc filth that passes for Modern “science”), which is anchored in objective truth, I don’t have opinions, I can only provide education.

On 2021-06-22, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> OTOH, if you are concerned about data in the database changing between
> the point in the VALIDATE block that checks a particular row, and the
> point the EXECUTE block that updates the relevant row, then yes, that
> is not catered for in the above description. However, that [and one
> other condition) is covered by Optimistic Locking.

Yes, that is what I meant.

>> >> OLAP Reports
>> > Why do you worry about locks in OLAP reports ?
>> > Who cares.
>> > Read locks are extremely short, and transient (held for the duration
>> > of the read operation [Statement], not the Transaction). We can’t control Read
>> > locks, even implicitly
>>
>> Are you saying that OLAP queries run in a lower isolation level (say,
>> READ COMMITTED)?
>
> Of course. That is what that ISOLATION LEVEL[ READ COMMITTED ] is there for.
>
> Not just OLAP queries, but all SELECTs outside a Transaction. That
> [ISOLATION LEVEL] is an ANSI SQL requirement, within the ACID
> requirement. We exclude [wait for completion of] uncommitted changes,
> but we do not need SERIALISED, in fact we want massively parallel, and
> we want the whole result set to be integral within itself (all
> internal dependencies intact and true).

What does "integral within itself" mean? Transactions at READ COMMITTED,
even if they are read-only, suffer from some anomalies.

> For understanding. Many systems in the MS world do their OLAP queries
> with READ UN-COMMITTED, to avoid Read Locks, but that means (a) the
> result set has no integrity, and (b) contains uncommitted changes
> which may disappear.

Correct. But queries at READ COMMITTED may view (committed) changes that
were not there when the transaction started. Do you accept that query
results may be off sometimes, or do you have some recipe to make them
avoid inconsistent reads and phantoms?

> For the MVCC world, they have no concept of READ
> COMMITTED/UNCOMMITTED, because they have only the false privatised
> version of the entire database, firmly secured between their big toes,
> which has only nominally “committed” changes, that has no relation to
> the version of other users. That nominally “committed” data blows up
> at COMMIT TRAN, just check the PooGres manuals re all the problems
> that are not supposed to happen in the Stonebraker fantasy, that
> happen, even to people who dutifully sing the mantra.

According to the manual
(https://www.postgresql.org/docs/current/transaction-iso.html), in
PostgreSQL each isolation level correctly prevents the anomalies that
are not allowed by the standard to occur at that level, and in some
cases it prevents more (e.g., READ UNCOMMITTED is like READ COMMITTED).
What are you referring to when you mention "all the problems that are
not supposed to happen... that happen"?

>> > But still, that is not Optimistic Locking. And in this post, there is
>> > nothing about Optimistic Locking.
>>
>> Agreed.
>
> Add Optimistic Locking, and the methods given in this thread would be complete.

Ok.

Nicola

Nicola

> On Thursday, 24 June 2021 at 07:59:50 UTC+10, Nicola wrote:
> > On 2021-06-22, Derek Ignatius Asirvadem wrote:

Sorry. Before I respond to the content, I need one clarification.

> > For understanding. Many systems in the MS world do their OLAP queries
> > with READ UN-COMMITTED, to avoid Read Locks, but that means (a) the
> > result set has no integrity, and (b) contains uncommitted changes
> > which may disappear.
>
> Correct. But queries at READ COMMITTED may view (committed) changes that
> were not there when the transaction started.

What “transaction started” ??? No transaction is involved.

The report connection would normally do SELECT [at] READ_COMMITTED which holds ReadLocks for the duration of the Statement, but in this abnormal case does SELECT [at] READ_UNCOMMITTED, holding no locks at all, and reading whatever is there on the disk, including uncommitted changes.

You might be stuck in the mindset that in order to read at { READ_UNCOMMITTED | READ_COMMITTED, } you must open a Transaction. No, that is the MV-non-CC insanity, the Stonebraker fantasy. No, that is not ACID. No, we do not do that.

As per ACID, the real ACID, not the perverted form that MV-non-CC pretends, we only need to open a Transaction for { REPEATABLE_READ | SERIALISABLE }.

(I have to stretch, and be very careful, when I am writing these things ... from my everyday position of never having those problems; that insanity; that mindset. The insanity is foreign to me, but I am aware of it. Otherwise I would make serious errors in understanding, and communication would be difficult (correcting simple errors). Please understand, you have to do the converse: from your everyday MV-non-CC insanity and fantasy, you need to think carefully about the Real ACID; Real Ordinary Locking; Real Transactions. Not the Straw Men, which are burning slowly.)

----------

Separately. What “rows that were not there [when the transaction started]” ??? How can a transaction change a row that is not there ??? Or even have knowledge about a row that is not there ???

Cheers
Derek

On 2021-06-24, Derek Ignatius Asirvadem <derek.asirvadem@gmail.com> wrote:
> Nicola
>
>> On Thursday, 24 June 2021 at 07:59:50 UTC+10, Nicola wrote:
>> > On 2021-06-22, Derek Ignatius Asirvadem wrote:
>
> Sorry. Before I respond to the content, I need one clarification.
>
>> > For understanding. Many systems in the MS world do their OLAP queries
>> > with READ UN-COMMITTED, to avoid Read Locks, but that means (a) the
>> > result set has no integrity, and (b) contains uncommitted changes
>> > which may disappear.
>>
>> Correct. But queries at READ COMMITTED may view (committed) changes that
>> were not there when the transaction started.
>
> What “transaction started” ??? No transaction is involved.

Ok, now that is clear.

> The report connection would normally do SELECT [at] READ_COMMITTED
> which holds ReadLocks for the duration of the Statement,

Ok. So, I am back to a question I have already posed to you:

- OLAP queries run at READ COMMITTED;
- Each OLAP query is a single SELECT statement;
- Each SELECT holds read locks for the duration of the statement.

But the duration of the statement may be significant (minutes, hours,
....) Don't such queries create contention (other transactions cannot
update the data read by the OLAP query until the query is over)?

> Separately. What “rows that were not there [when the transaction
> started]” ??? How can a transaction change a row that is not there
> ???

Of course it can't.

> Or even have knowledge about a row that is not there ???

Of course it can't. But it can see something that was not there before,
and it appears at a later time. I explain what I mean with an example:

Time Transaction Operation
0 T1,T2 start transaction;
1 T1 select sum(X) from R;
2 T2 insert into R(X) values (100);
3 T2 commit;
4 T1 select sum(X) from R;

At READ COMMITTED, T1 at time 4 will see the record inserted by T2 at
time 2, even if the record did non exist when T1 began. The SELECT at
time 1 and time 4 would then return different results.

Of course, this is not an issue if the only queries you run at READ
COMMITTED are single statements—provided that the system guarantees
statement-level consistency (it seems that some don't:
https://stackoverflow.com/questions/4772475/read-committed-isolation-level-in-sql-server-for-a-single-statement).

Nicola

On Thursday, June 24, 2021 at 7:33:00 PM UTC+10, Nicola wrote:
> On 2021-06-24, Derek Ignatius Asirvadem <derek.a...@gmail.com> wrote:
> > Nicola
> >
> >> On Thursday, 24 June 2021 at 07:59:50 UTC+10, Nicola wrote:
> >> > On 2021-06-22, Derek Ignatius Asirvadem wrote:
> >
> > Sorry. Before I respond to the content, I need one clarification.
> >
> >> > For understanding. Many systems in the MS world do their OLAP queries
> >> > with READ UN-COMMITTED, to avoid Read Locks, but that means (a) the
> >> > result set has no integrity, and (b) contains uncommitted changes
> >> > which may disappear.
> >>
> >> Correct. But queries at READ COMMITTED may view (committed) changes that
> >> were not there when the transaction started.
> >
> > What “transaction started” ??? No transaction is involved.
> Ok, now that is clear.
> > The report connection would normally do SELECT [at] READ_COMMITTED
> > which holds ReadLocks for the duration of the Statement,
> Ok. So, I am back to a question I have already posed to you:
>
> - OLAP queries run at READ COMMITTED;
> - Each OLAP query is a single SELECT statement;
> - Each SELECT holds read locks for the duration of the statement.
>
> But the duration of the statement may be significant (minutes, hours,
> ...) Don't such queries create contention (other transactions cannot
> update the data read by the OLAP query until the query is over)?
> > Separately. What “rows that were not there [when the transaction
> > started]” ??? How can a transaction change a row that is not there
> > ???
> Of course it can't.
> > Or even have knowledge about a row that is not there ???
> Of course it can't. But it can see something that was not there before,
> and it appears at a later time. I explain what I mean with an example:
>
> Time Transaction Operation
> 0 T1,T2 start transaction;
> 1 T1 select sum(X) from R;
> 2 T2 insert into R(X) values (100);
> 3 T2 commit;
> 4 T1 select sum(X) from R;
>
> At READ COMMITTED, T1 at time 4 will see the record inserted by T2 at
> time 2, even if the record did non exist when T1 began. The SELECT at
> time 1 and time 4 would then return different results.
>
> Of course, this is not an issue if the only queries you run at READ
> COMMITTED are single statements—provided that the system guarantees
> statement-level consistency (it seems that some don't:
> https://stackoverflow.com/questions/4772475/read-committed-isolation-level-in-sql-server-for-a-single-statement).
>
> Nicola

Hi Nicola,

If you're running a query in the READ COMMITTED isolation level then a shared lock is acquired for the duration of the read operation on whatever resource the lock protects. That resource may be a row or a page.
If you have acquired a row-level lock then the lock will be released once reading the row has finished.
If you have acquired a page lock then the lock will be released once reading the page has finished.

A query running for hours under READ COMMITTED would be locking and unlocking rows or pages as it encounters and processes them.

As for your second part concerning the sequence of events described, you are quite right. This is precisely the phenomenon that the REPEATABLE READ isolation protects against.
When you run a query with REPEATABLE READ isolation, shared locks are acquired and held for rows / pages until the transaction is committed or rolled back.
So if you then tried to modify data in those rows or pages, which involves acquiring an exclusive lock for those rows or pages, it would block - the exclusive lock being incompatible with the shared lock - thus allowing you to repeat the REPEATABLE READ and get the same result.

However, it's important to note that REPEATABLE READ isolation level does not acquire range locks (or predicate locks). This means that you can conceivably insert a new record that satisfies your REPEATABLE READ query's WHERE clause predicate.
For example, you could insert a new record that satisfies the REPEATABLE READ query predicate but ends up stored on a page that is not subject to a shared lock (the page might not have existed when you first ran the REPEATABLE READ query).
This is a phantom record, which SERIALIZABLE guards against using the aforementioned range (predicate) locks.

It's important to recognise that your REPEATABLE READ query - or any query for that matter - is executing in the context of a broader, shared, system. Seeing the outcome of newly committed transactions makes sense if you've chosen READ COMMITTED and understand the concurrency phenomenon that it protects against.

Importantly, and often overlooked as far as MVCC disadvantages go, the data you read in an hours-long query running under MVCC is arbitrarily old data.. If your query takes 2 hours to complete then the very last row processed by your query is potentially 2 hours out of date.

Daniel