How to shrink Oracle Context Index

On of databases I've found that one of Context Indexes is getting bigger and bigger everyday. It was around 5-10% grow per day for table DR$MY_TEXT_I$I which contains tokens(word). No of records indexed was almost the same, but index grown was unexpectedly big. It was due fact, that quite many rows were modified every day, and it caused re-indexing those rows during index synchronization (CTX_DDL.SYNC_INDEX). So I've decided to reclaim this space.

Method 1.

This was my first though:

• Drop Index ... /Create Index ...
• or Alter Index ... Rebuild

But those methods have some disadvantages:

• Users cannot perform text searches during this operation,
• It does not solve problem for longer time, as index will still grow ...

So I had to find root cause and eliminate it.

Method 2.

While looking for root cause I've find out, that Index has never been optimized, so it kept old (obsolete) data. My solution was:
SQL> exec ctx_ddl.optimize_index('MY_TEXT_I', 'FULL', 120);
SQL> alter table DR$MY_TEXT_I$I enable row movement;
SQL> alter table DR$MY_TEXT_I$I shrink space cascade;

With this method table DR$MY_TEXT_I$I took 30% of its original size.
So let me explain why this worked. With ctx_ddl.optimize_index context index was internally optimized. It means information about old documents were completely deleted and index was internally minimized. Remember - third parameter limits time (in minutes) allowed for optimization. If it is really big index it can take hours. But you can run this optimization many times, until your index will be fully optimized.
Then I've just shrunk table (shrink space cascade). But this operation requires to enable row movement on table first.

But this was one time operation. To avoid this problem in future I've scheduled job to run ctx_ddl.optimize_index on regular basis. Now this table has grown a little, but is no longer growing so fast.

Foot note:

• It was tested on Oracle EE 10gR2 (10.2.0.3),
• I'm not sure if row movement is supported by Oracle for Context Index table DR$<index>$I. It worked for me and I did not experience any problems since turning it on.
  

Cheers, Paweł

I've been tagged 3 times; 8 things about me

Few busy days without any blog reading and after I've gone through my feeds it occurred that I've been tagged 3 times:

1. By Eddie Awad
2. By H.Tonguç YILMAZ
   
3. By Jorrit Nijssen (Jornica)
   

Thanks You for tagging.
Now I feel that I really need to write something about me:

1. I was born and grow up in Krosno in south-east Poland (see satellite picture),
2. I 15th when Communism fall down in Poland. At this moment Poland is member of Schengen Agreement, and I can travel around Europe without passport. Lot of changes in last 19 years,
3. I'm Thawte Web of Trust Notary,
   
4. In 1993 I took 2nd place in Nation Contest in Programming for College Students. Unfortunately I was 6 month too old to take part in international final :(
5. In 1990 I've sold my first program; small DB written in Clipper. I did not think that this program will be used for so many years. In 1999 I had to do some changes to support year 2000,
6. My first program was written in Basic and was run on PSP-80 (page in Polish)
   
7. I'm married and we have 2 daughters (6 and 2 years old),
8. My favourite TV show is Top Gear with Jeremy Clarkson , Richard Hammond and James May.
   

I will not invite anyone directly as I do not want to accused of spam. All my readers that have blogs are welcome to post 8 things about themselves.

Cheers,
Paweł

Do not trust Google Maps

Ok, Might be title is too strong, but you should not trust Google Maps when searching for streets in Krosno, Poland (city where I was born). Here is sample, try to search for "Lwowska, 38-400 Krosno, Poland" and you get:
Search is working correctly, but names on street do not match reality. Instead of "Lwowska" street is "Jana Lenarta". Another example is "Platynowa" instead of "Podkarpacka". It seems that all street names are messed up on map. Hope they fix it soon.

Cheers, Paweł

Rant: Great fun with Wii

I'm not a computer game player, but Nintendo Wii really impressed me. I was with a visit to my friend and he showed me his new gadget. We started to play for a while, and time passed by. It's really addictive. The best thing is that you do not seat at computer and use joystick or keyboard, but you stand up and move whole body to play. It really gives pleasure, and we have good time. If you do not know Wii yet check this Wii commercial.

Have fun
Paweł

More on PHP with DRCP support

I was asked to compare DRCP to SHARED server connections. I've also promised to give some more statistics and comparisons for DRCP and standard connectivity.
First few results to compare to SHARED server as asked in comments to my previous post:

Version	Description	# Requests	Average	Median	Standard Deviation	Throughput
1	oci_connect DEDICATED server						Not tested
2	oci_pconnect DEDICATED server	1200	84	63	116	4077
3	oci_connect SHARED server	1200	280	264	150	1290
4	oci_pconnect SHARED server	1200	82	78	93	3965
5	oci_connect with DRCP	1200	104	94	45	3265
6	oci_pconnect with DRCP	1200	78	78	23	4396

In this case SHARED server is configured to use 4 dispatchers and max process = 1000.


Cheers, Paweł

Testing new OCI8 (1.3.0 beta) for PHP with DRCP Support

I've seen Christopher Jones announcement on availability of beta OCI8 extension for PHP with support for DRCP. I've decided to give it a try. On my test machine running OEL5 I've installed latest PHP 5.4.2 and then replaced OCI8 sources with new OCI8 1.3.0 beta release. I've configured PHP:
./configure --with-oci8=$ORACLE_HOME --with-apxs2=/usr/sbin/apxs --with-config-file-path=/etc/httpd/conf --enable-sigchild --without-sqlite --without-mysql
compiled it and installed. Everything went fine, and on sample page I've displayed phpinfo():

I've changed php.ini to enable Database Resident Connection Pooling:

oci8.connection_class="TESTPOOL"

and tested that it is used in PHP.
Then I've decided to test and compare performance of different methods of connection to Oracle from PHP. I've created simple PHP page, that connects to Oracle, execute simple query that returns one row, and inserts one row to another table. I've prepared test script for Apache JMeter against this page.
Test specification:

• Run on single processor machine
• Oracle and Apache+PHP run in VMware
• DRCP setup to have max 4 processes (3 processes effectively used, 4^th for authentication only)
• 6 simultaneous users opening the same web page 200 times
• 4 versions of PHP script were used:
  

1. oci_connect used to connect to Oracle without using DRCP (Database Resident Connection Pooling)– in this case new connection is created for every request from web browser. Connection is then closed at the end of script
   
2. oci_pconnect used to connect to Oracle without using DRCP – in this case new connection is created for every apache process that is responding to request from web browser. Connection is then closed at the end of script
   
3. oci_connect used to connect to Oracle with using DRCP (Database Resident Connection Pooling)– in this case for every request connection is obtained from pool and given back to pool at the end of script, seems that in this case only process is reused (PURITY=NEW), while session is not (see my observations when using SQL*Plus )
   
4. oci_pconnect used to connect to Oracle with using DRCP (Database Resident Connection Pooling)– in this case for every request connection is obtained from pool and given back to pool at the end of script. But in this case full potential of DRCP is used (PURITY=SELF)

Here are the results:

Version	Description	# Requests	Average	Median	Standard Deviation	Throughput
1	oci_connect without DRCP	1200	989	875	828	359
2	oci_pconnect without DRCP	1200	123	94	187	2505
3	oci_connect with DRCP	1200	117	94	115	2907
4	oci_pconnect with DRCP	1200	82	78	30	4060

I must say that I'm positively impressed by performance achieved by version 3 where oci_pconnect with DRCP was used. I expected that performance for version 2,3,4 will be very similar. But I've observed 60% increase in Throughput and 33% shorter response times when comparing version 2 and 4 (oci_pconnect with and without DRCP). I've changed one setting for my test in JMeter: I've turned on Keep-Alive for http transfer and here are the results:

Version	Description	# Requests	Average	Median	Standard Deviation	Throughput
1	oci_connect without DRCP						Not tested
2	oci_pconnect without DRCP	1200	84	63	116	4077
3	oci_connect with DRCP	1200	104	94	45	3265
4	oci_pconnect with DRCP	1200	78	78	23	4396

In this case I've observed less differences on Throughput and Response times between version 2 and 4. This is because with Keep-Alive causes that the same apache processes are used for requests coming from the same web browser. As connection is kept open, reduces time to establish connections between clients (web browsers) to apache server. Thats why we can see higher throughput. Additionally for oci_pconnect without DRCP less amount of sessions are created. Without Keep-Alive apache opened much more sessions, as it had to create more processes and every process had to connect to Oracle. Lets summarize my observations:

1. oci_pconnect with DRCP works stable in both cases (with and without Keep-Alive) giving maximun throughput and very low standard deviation,
2. oci_pconnect with DRCP has high standard deviation what means that there are lot of request with response time much higher then average,
3. In both cases connecting to Oracle with use of DRCP performed better than without.

Next time I'll try to go into extremes to see how DRCP behaves.
Cheers, Paweł

I've got spammed (in comments)

Due to quite massive spam I'm receiving today in comments I had to turn on word verification for comment posting. Hope it will protect me from spam for future. And sorry for inconvenience when commenting,

Cheers, Paweł