Data Virtualization Use Case - SATELLITE SPECTRUM AUCTION

                                First of Congratulations Bengaluru for the American Consulate. Americans love their Coffee and imagine Drones trying to deliver from Starbucks to American Consulate in Bengaluru. It's high time the Government of India considers auctioning Satellite Spectrum as it is pretty evident with the Russo-Ukraine war that if there is no control all will end at mayhem. Drones need the satellite internet more than ever to navigate from Point A to Point B but precise automated navigation needs to be implemented to avoid mid-air collision. How you ask?

                         Currently location based services use two parameters for precise coordinates locations namely X and Y Coordinate. Mid-Air Travel needs a third coordinate called Above Sea Level, Above Ground Level. The Z Coordinate. One important factor is simulating a route to avoid mid-air collissions no matter how accurately you design navigation routes. Come Data Virtualization to the rescue that can integrate data from multiple sources (navigation routes here) and there needs to be a simulation layer obviously with 100% accuracy of X,Y and Z Coordinates. 

THE GOVERNMENT OF INDIA SHOULD FORM A POLICY AROUND SATELLITE SPECTRUM AND IT'S AUCTION AS THERE HAS TO BE A GOVERNING BODY THAT GOVERNS ALL OUTCOMES OF SATELLITE SERVICES.

Imagine a Flying Taxi Drone travelling at 150 km/h in Bengaluru Traffic. Fastest Drone as of today Average Top Speed of 480.23 km/h - Link

OpenAI of Databases: Memory is cheap, Processing is Costly...

           As organizations gear up to implement Data Governance, Master Data Management and Meta Data Management Databases should AI themselves. Memory is cheap so think of every SQL Query being tracked in a central repository and so the unique row identifier of the SQL Query (DML and DQL) is stored in-memory. Irrespective of the number of joins, number of where conditions, analytic functions and so on.

Kind of Confusing ain't it. Let me explain. Think of this SQL Query. 

SELECT department_id, last_name, salary, RANK() OVER (PARTITION BY department_id ORDER BY salary) RANK FROM employees WHERE department_id = 60 ORDER BY RANK, last_name; 

              Obviously traditional databases would suggest to bitmap index deptartment_id column looking at cardinality for faster access. Indexing is nothing but storing the exact location of the row in memory (Data Object, Data Block, Data File and so on). Online Indexes are rebuilt online while the DDL is performed without the need for a DBA to rebuild indexes. 

        Now think of this. If the above SQL's result set originating rowid's (Employees Table Department No 60 as well as RANK function) is pre-populated in memory all the SQL Engine has to do is deal with rowid's for future instances of the same SQL Query. 

             In case of the above query when there is an update of an employee's department number all the SQL Engine has to do is update the SQL's result set rowid's (DML of rowid's for all SQL Query References that are stored in memory). Future instances of the same SQL Query will be rowid's processsing from memory. This will pave way for Real-Time ETL as well. Most importantly, OLTP to OLAP and OLTP DML resulting in rowid updates in memory that makes OLAP Querying Real-Time at the instant updates that flow from the OLTP System.

            Think of Mergers, Acquisitions and the amount of efforts spent in integration. Also think of Hyperlocal delivery where one warehouse can willfully work with multiple partners.

Until next time OpenAI of Databases: Memory is cheap, Processing is Costly...

Query Clause Indexing

Are the World Governments (Politicians) not spending money and borrowing money because of Global Indian Computing Standards?

Imagine being able to index any CLAUSE of the SQL Query...

I am not able to patent this as I have no Time and Money and on top of that no reply from Indian Patent Officers. 

The Query Clause Indexing can UNITE the WORLD for a Cause. 

Real-Time ETL

Blockbuster of Sorts in the making by INDIA...

FIRST MOVERS ADVANTAGE TOWARDS THE KOLAGATLA'S CAN CONTINUE TO SELL AND BUY IN THE USA DOLLARS PER SE...

SEMI-JOIN A Beautifully Architected Algorithm by TIBCO.

 An algorithm that is so amazing and felt like a hack in Data Virtualization.

A query in the clinical trials data of Astra Zeneca was taking around 2 hours to execute and a lot of caching errors.

It was called the clinical trials hub and I was tasked to optimize it. Semi-Join, Multi Table cache refresh along with drop and recreate indexes, the execution time of the refresh was 2 Minutes.

20 seconds - Execution of Query.
The rest of time - Insertion.

I was amazed because it had been long time since I got my head to work again but something kept running in my mind.

What if I would like to build something superior than the semi-join?

I would have to keep thinking for ever.

When there are two tables that are to be joined from Two Different Databases (Different DB Servers), SEMI-JOIN will convert the parent query in such a way the IN CLAUSE is re-written and the FETCH happens from a Single DataBase Server. I do not need to explain more as a person familiar with SQL will understand how this JOIN Algorithm works.