Australia Finance

Oct 17 2017

Bulk Processing in Oracle Part 1 – All Things Oracle #bulk #mail #processing


To learn is to change. Education is a process that changes the learner. Unknown

As you know (or may not know) there are two engines running in the Oracle database. The PL/SQL engine for executing all your PL/SQL code and the SQL engine for executing all you SQL statements. This means that every time your PL/SQL code needs data from the database, or writes to it control is passed from the PL/SQL runtime engine to the SQL. This process involves a lot of work which has to be done by the two engines, like writing the SQL statement in an area the SQL engine can access and having the SQL engine write the result in an area the PL/SQL engine can access. This is called a context-switch.

Reduce time spent

All these context-switches take time. Not too much time individually, which is why you don t see any slowdown of the program when you run a SQL statement that returns a single row, but when you run SQL statements in a loop then you actually perform a lot of SQL statements, which involves a lot of context switching. You actually want to reduce these context switches to a single one and bring back the entire resultset in one pass. This is where the Bulk processing capabilities of the Oracle database come in.

Bulk processing

Instead of fetching or updating a single row at a time since Oracle 8i it is possible to use the bulk features in the Oracle database. This means you can fetch an entire collection of rows in one roundtrip to SQL engine. Similarly you can insert, update or delete an entire collection in one roundtrip.

Bulk Collect

For fetching data there are a few things to consider. Instead of fetching a single value or row you are now fetching an entire set of values or rows. So instead of fetching into simple variables or records you must now fetch into a collection of scalars or a collection of records. See Collections in Oracle for some explanation of the different types of collections.

Consider the following code fragment:

The emp table (in the SCOTT schema) consists of 14 rows. That means this code will have to make 14 roundtrips to the database.

Let s change the code to utilize the Bulk Collect option.

Now we have just 1 roundtrip to the database. I may be that we have to code some more in PL/SQL, but our code runs a lot faster. If we declare an explicit cursor then every fetch in our code means a roundtrip to the database.

Consider the following code fragments. In the second fragment, changes that
have to be made are in bold.

We can base a collection type on the cursor record retrieved. Then we define a variable based on this collection type. We still have to open the cursor but we can now fetch all the records at once and bulk collect them into our collection. After having fetched all of the records, we have to write our own loop to process all the data. The collection used for bulk fetching is automatically initialized, extended (for Nested Tables and Varray) and densely filled, starting at index value 1.

Memory issues

If your table is really big and you bulk collect all the data into you collection you might run out of memory. Remember, your collection takes up space in the PGA. But you still can use the bulk collect feature, only in this case you want to LIMIT the rows retrieved on every roundtrip. You can limit the rows retrieved by adding the LIMIT clause to your fetch. By creating a loop you can still fetch all the rows from the database but in smaller batches. Please note that bulk collecting data will set the cursor %NOTFOUND to true when there are not enough rows to fetch, but it can still have fetched one or more rows. Bulk collect will also NOT raise the NO_DATA_FOUND exception when there is nothing found. This means you always have to check if there is anything retrieved before you start your processing. You should also check if the number of rows retrieved is the same as the limit you provided. If it is smaller, then you can exit the loop. Or check for the cursor %NOTFOUND flag, but do this at the end of your loop.

Patrick Barel is a PL/SQL Developer for AMIS Services in the Netherlands and an Oracle ACE. Besides working with SQL and PL/SQL did he co-develop CodeGen and wrote different plug-ins for PL/SQL Developer. He publishes articles on the AMIS Technology blog and on his own blog.

Written by admin

Leave a Reply

Your email address will not be published. Required fields are marked *