Rick's Blog » Oracle

RMAN backup error

Rick — Tue, 08 Feb 2011 02:09:45 +0000

… input archive log thread=2 sequence=46274 recid=166663 stamp=714935844 input archive log thread=2 sequence=46275 recid=166665 stamp=714935889 input archive log thread=2 sequence=46276 recid=166667 stamp=714939059 input archive log thread=2 sequence=46277 recid=166668 stamp=714939083 channel ch0: starting piece 1 at 29-MAR-10 released channel: ch0 RMAN-00571: =========================================================== RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS =============== RMAN-00571: =========================================================== RMAN-03009: failure of backup command on ch0 channel at 03/29/2010 18:27:07 ORA-19506: failed to create sequential file, name=”r2l9q6n6_1_1″, parms=”" ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: Communication with the server has not been iniatated or the server status has not been retrieved from the server. The fix was to remove the client identification from the rman script.

ORA-14400: inserted partition key does not map to any partition

Rick — Thu, 23 Dec 2010 00:29:49 +0000

Got this error in the emoms.trc log and thousands of .xml files in the recv directory. 2009-12-22 18:13:37,659 [XMLLoader0 30000031529.xml] ERROR emdrep.XMLLoaderContext flushTableStatements.1587 – Error while flushing insert statementjava.sql.SQLException: ORA-14400: inserted partition key does not map to any partition The fix was found on metalink as [ID 317046.1] Problem: Many Ora-14400 Errors Showing In Management Services And Repository Ora-14400 [ID 317046.1] SQL> exec emd_maintenance.analyze_emd_schema('SYSMAN'); PL/SQL procedure successfully completed. Also make sure your job_queue_processes is set to 10 or above. Hope that helps

RDBMS package: DBMS_AQ doesnt exist

Rick — Mon, 24 Aug 2009 00:08:00 +0000

While upgrading OEM from 10.2.0.4 to 10.2.0.5 I got the following error:[Text: Required RDBMS package: DBMS_AQ doesnt exist on this database. This package is needed for the proper functioning of the EM product. Please contact Oracle support to diagnose the RDBMS and once resolved, retry EM install.] This was caused by the sys password not being the same on all instances. The root cause was that a DBA had changed the sys password on a single instance to do some maintenance and did not change it back. This caused sys connections to the rac database to succeed and fail intermittently. The fix was to resync the sys password by running the following from sqlplus as the sysdba:SQL> alter user sys identified by “correct_password”;Once the passwords were correctly synced, the upgrade ran fine.

Setting the TNS_ADMIN envrionment

Rick — Mon, 06 Jul 2009 19:38:06 +0000

The TNS_ADMIN environment is used by the installation to find the location of the tnsnames.ora and the listener.ora files Default location The default location of the tnsnames.ora and listener.ora files are in $ORACLE_HOME/network/admin. If you want to change the location you have to show Oracle where to find the files. This is done by setting the TNS_ADMIN environment settings as follows: bash-3.00$ export TNS_ADMIN=/u01/app/oracle/network/admin If you’re using RAC, you’ll want to make sure that the Oracle Cluster Ready service knows where to find the file. This is important because OCR will be used to start/stop and manage your database. To setup bash-3.00$ srvctl setenv database -d rmant -T “TNS_ADMIN=/fs01/app/oracle/product/10.2.0/network/admin/” bash-3.00$ srvctl getenv database -d rmant TNS_ADMIN=/fs01/app/oracle/product/10.2.0/network/admin/

ORA-19506: failed to create sequential file

Rick — Fri, 03 Apr 2009 03:44:46 +0000

Many people run Oracle RMAN scripts from crontab and an OS shell script. We prefer to run all of our RMAN backups via OEM and the database scheduler instead of from a specific server or instance. There are many benefits OEM/database scheduler backups a few of which are: Centralized administration makes it easy to manage 100′s of databases Database affinity means that backups run even if a server crashes OEM monitors the output of the RMAN and can alert the DBA if there are errors One thing that we have noticed is that when we used our OS (server) RMAN shell scripts we had a string similar to the following: send ‘NB_ORA_CLIENT=gmrac1 This parameter is optional for Networker and is used to assist Networker with recovery operations to alternate servers. Occasionally, the RMAN would fail for no immediately obvious reason. On closer inspection we saw Networker was timing out while connecting to the target server. Everytime the database RMAN job was launched from gmrac1 it would succeed. However when it was run from any other server in the cluster the job would fail with the following: RMAN-03009: failure of backup command on ch0 channel at 02/05/2009 16:51:46 ORA-19506: failed to create sequential file, name=”mqk6k505_1_1″, parms=”"ORA-27028: skgfqcre: sbtbackup returned error ORA-19511: Error received from media manager layer, error text: VxBSACreateObject: Failed with error: Server Status: Communication with the server has not been iniatated or the server status has not been retrieved from the server. [sic] The fix was to remove the optional parameter NB_ORA_CLIENT=gmrac1 and let Networker do the right thing. For reference, our broken RMAN script is shown below: run { sql ‘alter system archive log current’; allocate channel ch0 type ‘SBT_TAPE’; send ‘NB_ORA_CLIENT=gmrac1,NB_ORA_POLICY=PROD_ora, NB_ORA_SERV=backup.betterlogic.com’; backup tag “prod_archivelogs_only” archivelog all not backed up; delete noprompt archivelog until time ‘sysdate -2′ backed up 1 times to ‘SBT_TAPE’; release channel ch0;}

Oracle Enterprise Manager Grid Control

Rick — Mon, 12 May 2008 21:18:54 +0000

EM can automatically connect to Metalink to check for required patches for each target database and download them to a software library. Over the next few months I will add monitoring, alerts, admin accounts, developer user accounts (so they can get their own ADDM (statspack) results), and groups. I will also work to add every db to the Grid. This is just the start. EM is pretty cool and I expect we will be able to: * Increase uptime and proactive management * Automate the instance reporting (for licensing contract purposes) * Improve general and custom reporting for all instances * Centralize much of the management * Compare database settings and schemas between two databases or between a stored baseline and any database * Create new users in multiple databases using any other user as a starting template * Handle configuration management * Assist with change control * Keep configuration histories * Make it simple to answer questions like: o “What was the database doing yesterday at 2:22PM?” o “When was this database last backed up?” o “Are their any blocking locks?” o “What are the top SQL queries?”, “What are the top five wait events?” o “What hosts are connected to my database?” (i.e. are development boxed connecting to production databases) o “What is the explain plan for this query?” * Job queuing and command execution library More to follow… -Rick

Rman recovery example

Rick — Sat, 05 Apr 2008 22:51:19 +0000

Recovery example SQL> select status, enabled, online_time, name from v$datafile order by name; STATUS ENABLED ONLINE_TI ------- ---------- --------- NAME -------------------------------------------------------------------------------- RECOVER READ WRITE 13-AUG-07 +DATA01/cp04/datafile/bb_bb60_indx.281.630546303 ONLINE READ WRITE 13-AUG-07 +DATA01/cp04/datafile/bb_bb60_stats_data.282.630546385 SQL> exit Disconnected from Oracle Database 10g Enterprise Edition Release 10.2.0.3.0 - 64bit Production With the Partitioning, Real Application Clusters, OLAP and Data Mining options bash-3.00$ rman Recovery Manager: Release 10.2.0.3.0 - Production on Sat Apr 5 18:09:42 2008 Copyright (c) 1982, 2005, Oracle. All rights reserved. RMAN> connect target connected to target database: CP04 (DBID=429165704) RMAN> sql 'alter database datafile 9 offline'; using target database control file instead of recovery catalog sql statement: alter database datafile 9 offline RMAN> recover datafile 9; Starting recover at 05-APR-08 allocated channel: ORA_DISK_1 channel ORA_DISK_1: sid=50 instance=cp041 devtype=DISK starting media recovery media recovery complete, elapsed time: 00:00:06 Finished recover at 05-APR-08 RMAN> sql 'alter database datafile 9 online'; sql statement: alter database datafile 9 online RMAN>

Schedule incidental backups via at

Rick — Fri, 21 Dec 2007 03:18:00 +0000

This little at job performs a full backup and sends the output of the log file to me via email. I use this for incidental one-time backups of databases for releases and migrations. I like to schedule these early enough that I can get the results of the logfile and create another backup if this one fails for any reason. Notice the -m and the cat; they make sure I get email as soon as the task is complete. -bash-3.00$ at -m 21:00 tomorrowat> /export/home/users/oracle/com.r.cron.betterlogic.full at> cat \'ls -rt /export/home/users/oracle/rman-logs/*full*|tail -1\'at> commands will be executed using /bin/bash job 1198288801.a at Fri Dec 21 21:00:00 2007

Two important Daylight Savings Time dates to remember (October 28th and November 4th)

Rick — Thu, 18 Oct 2007 04:00:00 +0000

FYI – The new Daylight Saving Time changes that started in March 2007 will soon be changing back to Standard Time on November 4th, 2007 . Year DST Begins DST Ends 2007 March 11 November 4 2008 March 9 November 2 2009 March 8 November 1 2010 March 14 November 7 2011 March 13 November 6 2012 March 11 November 4 2013 March 10 November 3 2014 March 9 November 2 2015 March 8 November 1 Additionally remember that by the old definition DST ended the last Sunday in October. In 2006 daylight time ended on October 29, 2006. In 2007 however, daylight time would have ended on October 28, 2007. Why do we care? Because any machines or services that did not get the correct DST patches will probably try to change to standard time on the October 28th date. So be on the watch-out for oddities on both October 28th and November 4th. http://tf.nist.gov/general/history.htm

Sync passwords from one database to another

Rick — Sat, 22 Sep 2007 14:00:00 +0000

Here is a simple way to syncronize schema passwords from one Oracle database to another. On the source database run the following sqlplus command: SQL> select ‘alter user ‘ || username || ‘ identified by values ”’|| password ||”’;’from dba_users where username in (‘GMADMIN’,'GM_0′,’GM_0_REPORT’,'GM_0_STATS’); That will produce the following output which you can run on your new database: ‘ALTERUSER’||USERNAME||’IDENTIFIEDBYVALUES”’||PASSWORD||”’;’——————————————————————————–alter user GM_0_REPORT identified by values ‘SJINE200223′;alter user GM_0_STATS identified by values ‘SKDNO9S9S’;alter user GM_0 identified by values ‘SD8HF872B34′;alter user GMADMIN identified by values ‘SN828WWHB88′;