Oracle Database -External Table – How to populate/ outbound file creation using External table


As you know we can load data from OS flat files into the Oracle environment,Now here showing how to create an external file / populating external table

Lets take a simple example: –

Step 1: – Create a directory

SQL> create directory my_dir as ‘/usr/abc’; —

Make sure Oracle OS user hasprivilege to write on this directory.

Step 2: – Grant permission to user SCOTT

SQL> grant read,write on directory my_dir to scott;

Step 3: – Create the External Table:

SQL> CREATE TABLE scott.extract_emp_dept

ORGANIZATION EXTERNAL

(TYPE ORACLE_DATAPUMP DEFAULT DIRECTORY my_dir

LOCATION (’emp.exp’))

reject limit unlimited

AS

select e.ename,d.dname from emp e, dept d where

e.deptno=d.deptno;

SQL> select * from scott.extract_emp_dept;

And you will find a file generated i.e. ’emp.exp’ in /usr/abc directory. Now you can take this file to the target database and create an external table and associate the file with that table.

Step 4: – to be performed on the source database

SQL> set long 2000

SQL> Select dbms_metadata.get_ddl(‘TABLE’,’EXTRACT_EMP_DEPT’) from dual;

The above command will give you the DDL command to create the table at target database.

Advertisements

Oracle Database : External Tables Overview and Usage


What are External Tables?

External tables are like regular SQL tables with some exceptions:
          – The metadata of external tables is created using the SQL "CREATE TABLE … ORGANIZATION EXTERNAL" statement. 
          – The data resides outside the database in OS files, thus the EXTERNAL organization.
          – The OS files are identified inside the database through a logical directory defining the OS physical directory where they are located. The owner of the files is Oracle. Users granted access via the Database can access the files through the external table. 
          – The data is read only. 
          – You cannot perform any DML operations, nor create indexes. 
          – The external table can be queried and joined directly, in parallel using the SQL statement SELECT.

Example
——— 
A. Create the external table to extract the data without loading them inside the database

1. Create flat files emp1.dat

2 Create a directory that defines the location of the directory where the flat files reside on the OS
3 Create the external table (metadata)
4 Select data from the external table to verify that data are visible

  • Create flat files emp1.dat

$ vi /u01/ora9i/data/emp1.dat
7369,SMITH,CLERK,7902,17-DEC-80,100,0,20
7499,ALLEN,SALESMAN,7698,20-FEB-81,250,0,30
7521,WARD,SALESMAN,7698,22-FEB-81,450,0,30
7566,JONES,MANAGER,7839,02-APR-81,1150,0,20

Check OS level permissions and ensure the user has read/write privileges to this directory.

  • Create a logical directory to map the OS directory where the external files reside and grant permissions to SCOTT
  • SQL> create directory emp_dir as ‘/u01/ora9i/data’ ;
    Directory created.
    SQL> GRANT READ ON DIRECTORY emp_dir TO scott;
    Grant succeeded.
    SQL> GRANT WRITE ON DIRECTORY emp_dir TO scott;
    Grant succeeded.

  • Create the external table :
  • SQL> create table scott.emp_ext
    (emp_id number(4), ename varchar2(12),
    job varchar2(12) , mgr_id number(4) ,
    hiredate date, salary number(8), comm number(8),
    dept_id number(2))
    organization external
    (type oracle_loader
    default directory emp_dir
    access parameters (records delimited by newline
    fields terminated by ‘,’)
    location (’emp1.dat’);
    Table created.

  • Check the data retrieval:

SQL> select * from scott.emp_ext;

EMP_ID ENAME    JOB        MGR_ID  HIREDATE  SALARY  COMM  DEPT_ID
—— ——-  ——–   ——- ——— ——  —– ——-
7369   SMITH    CLERK      7902    17-DEC-80 100     0     20
7499   ALLEN    SALESMAN   7698    20-FEB-81 250     0     30
7521   WARD     SALESMAN   7698    22-FEB-81 450     0     30
7566   JONES    MANAGER    7839    02-APR-81 1150    0     20
7654   MARTIN   SALESMAN   7698    28-SEP-81 1250    0     30
7698   BLAKE    MANAGER    7839    01-MAY-81 1550    0     30
7934   MILLER   CLERK      7782    23-JAN-82 3500    0     10
7 rows selected.

Oracle 10g Cost Based Optimizer


Hello Friends, here i like to give some brief on Cost based optimizer in Oracle, i am not covering Rule based optimizer as it is not much in use as well not recommended. plz share ur feedback or if u like to contribute more on this topic,

Oracle 10g Cost Based Optimizer

Whenever you execute a SQL statement, a component of the database known as the optimizer must decide how best to access the data operated on by that statement. To figure out the optimal execution path for a statement, the optimizers consider the following:

  • The syntax you’ve specified for the statement
  • Any conditions that the data must satisfy (the WHERE clauses)
  • The database tables your statement will need to access
  • All possible indexes that can be used in retrieving data from the table
  • The Oracle RDBMS version
  • The current optimizer mode
  • SQL statement hints
  • All available object statistics (generated via the ANALYZE command)
  • The physical table location (distributed SQL)
  • INIT.ORA settings (parallel query, async I/O, etc.)

Understanding the Cost-Based Optimizer

The information required by the cost-based optimizer is available once a table has been analyzed via the ANALYZE command, or via the DBMS_STATS facility. Plz note The ANALYZE command and the DBMS_STATS functions collect statistics about tables, clusters, and indexes, and store those statistics in the data dictionary.

The RDBMS kernel defaults to using the cost-based optimizer under a number of situations, including the following:

  • OPTIMIZER_MODE = CHOOSE has been specified in the INIT.ORA file, and statistics exist for at least one table involved in the statement
  • An ALTER SESSION SET OPTIMIZER_MODE = CHOOSE command has been executed, and statistics exist for at least one table involved in the statement
  • An ALTER SESSION SET OPTIMIZER_MODE = FIRST_ROWS (or ALL_ROWS) command has been executed, and statistics exist for at least one table involved in the statement
  • A statement uses the FIRST_ROWS or ALL_ROWS hint (e.g., SELECT /*+ FIRST_ROWS */. . .)

ANALYZE command

The way that you analyze your tables can have a dramatic effect on your SQL performance. If your DBA forgets to analyze tables or indexes after a table re-build, the impact on performance can be devastating. If your DBA analyzes each weekend, a new threshold may be reached and Oracle may change its execution plan. The new plan will more often than not be an improvement, but will occasionally be worse.

If you do want to analyze frequently, use DBMS_STATS.EXPORT_SCHEMA_STATS to back up the existing statistics prior to re-analyzing. This gives you the ability to revert back to the previous statistics if things screw up. When you analyze, you can have Oracle look at all rows in a table (ANALYZE COMPUTE) or at a sampling of rows (ANALYZE ESTIMATE). Typically, When a table is analyzed using ANALYZE, all associated indexes are analyzed as well. If an index is subsequently dropped and recreated, it must be re-analyzed. Be aware that the procedures DBMS_STATS.GATHER_SCHEMA_STATS and GATHER_TABLE_STATS analyze only tables by default, not their indexes. When using those procedures, you must specify the CASCADE=>TRUE option for indexes to be analyzed as well.

Following are some sample ANALYZE statements:

ANALYZE TABLE EMP ESTIMATE STATISTICS SAMPLE 5 PERCENT FOR ALL INDEXED COLUMNS;

ANALYZE INDEX EMP_NDX1 ESTIMATE STATISTICS SAMPLE 5 PERCENT FOR ALL INDEXED COLUMNS; ANALYZE TABLE EMP COMPUTE STATISTICS FOR ALL INDEXED COLUMNS;

If you analyze a table by mistake, you can delete the statistics. For example:

ANALYZE TABLE EMP DELETE STATISTICS;

Analyzing can take an excessive amount of time if you use the COMPUTE option on large objects. We find that on almost every occasion, ANALYZE ESTIMATE 5 PERCENT on a large table forces the optimizer make the same decision as ANALYZE COMPUTE.

Inner workings of the cost-based optimizer

The cost-based optimizer is flexible and can adapt to its environment. This adaptation is possible only once the necessary underlying object statistics have been refreshed (re-analyzed). What is constant is the method by which the cost-based optimizer calculates each possible execution plan and evaluates its cost (efficiency).

The cost-based optimizer’s functionality can be (loosely) broken into the following steps:

1.       Parse the SQL (check syntax, object privileges, etc.).

2.       Generate a list of all potential execution plans.

3.       Calculate (estimate) the cost of each execution plan using all available object statistics.

4.       Select the execution plan with the lowest cost.

The cost-based optimizer will be used only if at least one table within a SQL statement has statistics (table statistics for unanalyzed tables are estimated). If no statistics are available for any table involved in the SQL, the RDBMS will resort to the rule-based optimizer, unless the cost-based optimizer is forced via statement-level HINTS or by an optimizer goal of ALL_ROWS or FIRST_ROWS.

To understand how the cost-based optimizer works and, ultimately, how to exploit it, we need to understand how it thinks.

Primary key and/or UNIQUE index equality —A UNIQUE index’s selectivity is recognized as 100%. No other indexed access method is more precise. For this reason, a unique index is always used when available.

Non-UNIQUE index equality —For non-UNIQUE indexes, index selectivity is calculated. The cost-based optimizer makes the assumption that the table (and subsequent indexes) have uniform data spread unless you use the FOR ALL INDEXED COLUMNS option of the ANALYZE. That option will make the cost-based optimizer aware of how the data in the indexed columns is skewed.

Range evaluation —For index range execution plans, selectivity is evaluated. This evaluation is based on a column’s most recent high-value and low-value statistics. Again, the cost-based optimizer makes the assumption that the table (and subsequent indexes) have uniform data spread unless you use the FOR ALL INDEXED COLUMNS option when analyzing the table.

Range evaluation over bind variables—For index range execution plans, selectivity is guessed. Prior to Oracle9i, because bind variable values are not available at parse time (values are passed to the cursor after the execution plan has been decided), the optimizer cannot make decisions based on bind variable values. The optimizer assumes a rule of thumb of 25% selectivity for unbounded bind variable ranges (e.g., WHERE dept_no = :b1) and 50% selectivity for bounded ranges (WHERE dept_no > :b1 AND dept_no < :b2). Beginning with Oracle9i, the cost-based optimizer obtains bind variable values prior to determining an execution plan.

System resource usage —By default, the cost-based optimizer assumes that you are the only person accessing the database. Oracle9i gives you the ability to store information about system resource usage, and can make much better informed decisions based on workload (read up on the DBMS_STATS.GATHER_SYSTEM_STATS package).

Current statistics are important —The cost-based optimizer can make poor execution plan choices when a table has been analyzed but its indexes have not been, or when indexes have been analyzed but not the tables.

DataLoad with Oracle E-business Suites (Oracle Applications/ Financials)


DataLoad with Oracle E-business Suites (Oracle Applications/ Financials)

 

 

 

 

 

DataLoad is a tool for manipulating the data and commands in Oracle E-Business and other software by sending pre-defined data and commands to the target program. These help pages are the primary source of information about DataLoad and how to use it. Follow one of the menu links on the left to the section which best fits your question or use the table of contents to find the information you are looking for. Implementing or supporting any application often requires significant effort to configure and load master and transaction data, for example from legacy systems, an expensive and time-consuming activity.

DataLoad reduces the time and costs associated with data migration. Its easy-to-use interface means that it can be used by non-technical employees, giving end-users an active role in the implementation and support of any system.

·   Non-technical users can easily configure and load data into any system

·   Recommended by Oracle, DataLoad contains specific additional features to load large volumes of data in any implementation of Oracle Applications.

·   DataLoad is sponsored by leading UK systems integrator Edenbrook, which has committed to support and improve the tool.

·   DataLoad is available in Classic and Professional editions, catering to all types of application implementation from the simple to the vastly complex.

·   DataLoad Classic is available to download free of charge.

 

 

 

Q. Can I use DataLoad with Oracle E-Business 11i, E-Business R12, etc?

A. DataLoad works with any Windows based applications and contains enhancements for use with Oracle E-Business. DataLoad has been tested with all current versions of E-Business that are supported by Oracle and has been proved to work correctly. As new versions of E-Business are released DataLoad is tested with these releases to ensure continued support. Full R12 compatibility was added in DataLoad V4.3.9.0.

 

Oracle – Technical FAQ – Part 4


 Q18.) What are the OOPS concepts in Oracle?

A18.) Oracle does implement the OOPS concepts. The best example is the Property Classes. We can categorise the properties by setting the visual attributes and then attach the property classes for the objects. OOPS supports the concepts of objects and classes and we can consider the property classes as classes and the items as objects.

 

Q19) What is the difference between candidate key, unique key and primary key?

A19) Candidate keys are the columns in the table that could be primary keys and the primary key is the key that has been selected to identify the rows. Unique key is also useful for identifying the distinct rows in the table.

 

Q20) What is concurrency?

A20) Concurrency is allowing simultaneous access of same data by different users. Locks useful for accessing the database are

a) Exclusive — The exclusive lock is useful for locking the row when an insert , update or delete is being done. This lock should not be applied when we do only select from the row.

b) Share lock — We can do the table as Share_Lock as many share_locks can be put on the same resource.

 

Q21) What are the Previleges and Grants?

A21) Privileges are the right to execute a particular type of SQL statements. The grant has to be given by the owner to object.

 

Q22) What are Table Space, Data files, parameter file, control files?

A22) Table space: The table space is useful for storing the data in the database. When a database is created two table spaces are created.

a) System Table space : This data file stores all the tables related to the system and dba tables.

b) User Table Space: This data file stores all the user related tables.

We should have separate table spaces for storing the tables and indexes so that the access is fast.

Data files : Every oracle Data base has one or more physical data files. They store the data for the database. Every datafile is associated with only one database. Once the data file is created the size can not change. To increase the size of the database to store more data we have to add data file.

Parameter file:Parameter file is needed to start an instance. A parameter file contains the list of instance configuration parameters e.g. db_block_buffers = 500

db_name = ORA7

db_domain = u.s.acme lang

Control files : Control files record the physical structure of the data files and redo log files . They contain the DB name, name and locations of dbs , data files, redo log files and time stamp.

 

Q23) What is physical storage of the Data?

A23) The finest level of granularity of the data base are the blocks.

Extent : Extent is the number of specific number of contigious data blocks.

Segments : Set of Extents allocated for Extents. There are three types of segments

a) Data Segment : Non clustered tables has data segment data of every table is stored in cluster data segment.

b) Index Segment : Each index has index segment that stores data.

c) Rollback Segment : Temporarily store ‘undo’ information.

 

Q24) What is Normalisation ? Explain 1NF, 2NF, 3NF.

Normalization is a process of eliminating data redudancy.

1NF- No repeating groups.

2NF- Eliminates the functional dependency on the partial key field.

3NF- Eliminates the functional dependency on the non-key field.

Oracle – Technical FAQ – Part 2


Q6) What are the database triggers and stored procedures?
A6) Database triggers are procedures that are automatically executed as a result of insert in, update to or delete from table.
Database triggers have the values old and new to denote the old value in the table before it is deleted and the new indicated the new value that will be used. DT are useful for implementing complex business rules which can not be enforced using the integrity rules. We can have the trigger as before trigger or after trigger and at statement or row level.
e.g. operations insert, update, delete — 3
before, after 3 * 2 — a total of 6 combinations

At statement level (once for the trigger) or row level (for every execution) 6 * 2 — a total of 12.

Thus a total of 12 combinations are there and the restriction of usage of 12 triggers has been lifted from Oracle 7.3 onwards

Stored Procedures : Stored procedures are procedures that are stored in complied form in the database. The advantage of using the stored procedures is that many users can use the same procedure in compiled and ready to use format.

 

Q7) How many integrity rules are there and what are they?

A7) There are three integrity rules and they are as follows

a)Entity integrity Rule : The entity integrity rule enforces that the primary key can not be Null

b) Foreign key integrity rule : The FKIR denotes that the relationship between the foreign key and the primary key has to be enforced. When there is data in child tables the master tables can not be deleted.

c) Business integrity rules : The third integrity rule is about the complex business processes which can not be implemented by the above 2 rules.

 

Q.8.) What are the various Master and Detail relationships?

A.8.) The various Master and Detail relationship are

a) Non Isolated : The Master can not be deleted when a child is existing

b) Isolated : The master can be deleted when the child is existing

c) Cascading : The child gets deleted when the master is deleted.

 

Q9) What are the various Block Coordination properties ?.

A9) The various block coordination properties are

a) Immediate – Default setting. The detail records are shown when the Master Record are shown.

b) Differed with AutoQuery — Oracle forms defer fetching the detail records until the operator navigates to the detail block.

C) Deffered with no auto query — The operator must navigate to the detail block and explicitly executes a query.

 

Q10) What are the different Optimisation techniques ?

A10) The various optimisation techniques are

a) Execute plan : We can see the plan of the query and change it accordingly based on the indexes

b) Optimizer_hint : Set_item_property (‘DeptBlock’,OPTIMIZER_HINT,’FIRST_ROWS’);

Select /* + First Rows */ Deptno, Dname, Loc, Rowid from dept

where (Deptno > 25)

c) Optimize_sql : By setting the optimize_sql = No, Oracle Forms assign a single cursor for all SQL statements. This slow downs the processing because for everytime the SQL must be parsed whenever they are executed. f45run module = my_firstform useid = scott/tiger optimize_Tp = No

 

  more questions are here : https://knoworacle.wordpress.com/2008/06/08/oracle-technical-faq-part-2/

Part 3 is here :   https://knoworacle.wordpress.com/2008/06/09/oracle-technical-faq-part-3/

Part 1 is here : https://knoworacle.wordpress.com/2008/08/11/oracle-technical-faq-part-1

 

 

 

Oracle – Technical FAQ – Part 1


Q1) What are the background processes in Oracle and what are they?
Ans: This is one of the most frequently asked question. There are basically 9 processes but in a general system we need t mention the first five background processes. They do the house keeping activities for the Oracle and are common in any system
The various background processes in oracle are
a) Data base writer (DBWR) : Data Base Writer writes modified blocks from databas buffer cache to Datafiles. This is required since the data is not written whenever a transaction is commited.

b) LogWriter(LGWR) : LogWriter writes the redo log entries to disk. Redo log data is generated in redo log buffer of SGA. As transaction commits and logs buffer fills, LGWR writes log entries into a online redo log file.

c) System Monitor (SMON) : The System Monitor performs instance recovery at instance startup. This is useful for recovery from system failure.

d) Process Monitor (PMON) : The Process Monitor performs process recovery when user process fails. Pmon clears and frees resources that process was using.

e) CheckPoint (CKPT): At specified times, all modified database buffers in SGA are written to data files by DBWR at checkpoints and updating all data files and control files of database to indicate the most recent checkpoint.

f) Archiever (ARCH) : The archiever copies online redo log files to archival storal when they are busy.

g) recoveror (RECO) : The recoveror is used to reslove the distributed transaction in network.

h) Dispatcher (Dnnn) : The Dispatcher is useful in Multi Threaded Architecture.

i) Lckn : We can have upto 10 lock processed for inter instance locking in parallel sql.

Q2) How many types of sql statements are there in Oracle?

A2) There are basically 6 types of sql statements. They are

a) Data defination Language (DDL) : The DDL statements define and maintain objects and drop objects.

b) Data Manipulation Language (DML) : The DML statements manipulate database data

c) Transaction Control Statements : Manage change by DML

d) Session Control : Used to control the properties of current session enabling and disabling roles and changing e.g. Alter statements , Set Role

e) System Control Statements : Change properties of Oracle Instance e.g. Alter System

f) Embedded Sql : Incorporate DDL, DML and TCS in processing Languagel e.g. Using the SQL statements in language such as ‘C’, Open, Fetch, Execute and close

Q3) What is a transaction in Oracle ?

A3) A transaction is a logical unit of work that compromises one or more SQL statements executed by a simple user. According to ANSI, a transaction begins with first executable statement and ends when it is explicitly commited or rolled back.

Q4) What are the key words used in Oracle?

A4) The key words that are used in Oracle are

a) Commiting : A transaction is said to be commited when the transaction makes permanent changes resulting from the SQL statements

b) Rollback : A transaction that retracts any of the changes resulting from SQL statements in Transaction.

c) Savepoint : For long transactions that contain many SQL statements , intermediate markers or savepoints are declared. Savepointes can be used to divide a transaction into smaller points.

d) Rolling forward : Process of applying redo log during recovery is called rolling forward.

e) Cursor : A cursor is a handle (name or a pointer) for the memory associated with a specific statement. A cursor is basically an area allocated by Oracle for executing the SQL statement. Oracle uses an implicit cursor statement for single row query and uses Explicit cursor for a multirow query.

f) System Global Area (SGA) : The SGA is a shared memory region allocated by the Oracle that contains Data and control information for one Oracle Instance. It consists of Database buffre cache and redo log buffer.

g) Program Global Area (PGA) : The PGA is a memory buffer that contains data and control information for server process.

h) Database Buffer Cache : Database buffer of SGA stores he most recently used blcoks of database data. The set of database buffers in an instance is called Database Buffer Cache.

i) Redo log buffer : Redo log buffer of SGA stores all the redo log entries.

j) Redo log files :Redo log files are set of files that protect altered database in memory that has been not written to Data files. They are basically used for backup when a database crashes.

k) Process : A process is a ‘thread of control’ or mechanism in Operating System that executes series of steps

Q5) What are procedures, functions and packages?

A5) Procedures and functions consist of set of PL/SQL statements that are grouped together as a unit to solve a specific problem or perform a set of related tasks.

Procedure do not return values while functions return one and only one value.

Packages provide a method of encapsulating and storing related procedures, functions and other package contents.

 more questions are here : https://knoworacle.wordpress.com/2008/06/08/oracle-technical-faq-part-2/

Part 3 is here :   https://knoworacle.wordpress.com/2008/06/09/oracle-technical-faq-part-3/