Using 4k Redo Logs on Flash, 4k-Disk and SSD-based Storage (Doc ID 1681266.1)
Using 4k Redo Logs on Flash, 4k-Disk and SSD-based Storage (Doc ID 1681266.1)
Many modern storage systems, especially those based on flash memory or SSDs, conform to an industry standard known as Advanced Format. This standard was created with a view to increasing the sector size (i.e. the minimum-possible I/O size) of storage devices from its historical value of 512 bytes to the more efficient value of 4096 bytes (or 4KB).
The Advanced Format standard describes two modes of operation. The first is native mode (known as “4K”, which presents all physical and logical blocks exactly as they are, i.e. as units of 4096 bytes. The drawback of native mode is that operating systems and software such as databases have to be capable of performing I/O operations using this new method. While Oracle Database and Grid Infrastructure have supported 4K devices since version 11g Release 2, many operating systems still do not fully support 4K while others require later versions of the kernel (Linux, for example, requires kernel version 2.6.32 or later). Note: for engineered systems including ODA, follow their own best practice.
The alternative mode of operation is emulation mode (known as “512e”). In 512e mode, physical blocks on storage are still 4096 bytes but are presented as a number of logical blocks with sizes of 512 bytes, resulting in full backwards-compatibility with all operating systems and software. However, since underlying physical I/Os must still be issued in multiples of 4KB, this presents a new potential issue.
Partial I/O
If, in emulation mode, an I/O is issued of size 512 bytes, the underlying storage platform must service this with an I/O of 4KB which is therefore eight time larger. This is known as a partial I/O. If many partial I/Os are issued, the underlying storage platform may be driven to perform substantially more I/O than will be seen from the host. Moreover, if the I/O is a 512 byte write operation, the entire 4096 byte physical block must be read, then the updated 512 byte block changed, followed by the amended 4096 byte block being written. This additional work can result in increased latency from storage platforms and result in poor performance.
Customers using Advanced Format storage in emulation mode therefore need to consider which objects are capable of generating large numbers of I/Os with sizes smaller than 4096 bytes. Partial I/O from datafiles can be avoided by using a database block size which is a multiple of 4KB (such as the default setting of 8KB), while database control files use a 16KB block size as standard. The most common cause of partial I/O however, is the online redo log.
Online Redo Logs
By default, database online redo log files have a block size of 512 bytes. From Oracle Database version 11.2 this can be changed with the BLOCKSIZE clause to values of 512, 1024 or 4096. For example:
ALTER DATABASE ADD LOGFILE GROUP 5 SIZE 100M BLOCKSIZE 4096;
However, when creating the redo log, Oracle issues a check to ensure the underlying device has a matching sector size. For a native mode device, the check would succeed and the redo log would be created. But for an emulation mode device, which is effectively pretending to be 512 bytes, the check fails with a message of similar form to:
ORA-01378: The logical block size (4096) of file +DATA is not compatible with the disk sector size (media sector size is 512 and host sector size is 512)
If it is known that the underlying storage actually uses a 4KB sector size, this check can be overridden by setting the underscore parameter _disk_sector_size_override to true within the database instance (the parameter can be changed dynamically):
ALTER SYSTEM SET “_DISK_SECTOR_SIZE_OVERRIDE”=”TRUE”;
At this point online redo logs can be created with the desired 4096 byte block size and any unwanted 512 byte redo logs can be dropped.
Misaligned I/O
One final point to consider with the use of emulation mode systems is that care should be taken to ensure all database objects are aligned on the boundary of 4KB physical sectors. If not, misaligned I/O can result, where a single I/O call actually causes two partial I/Os across adjacent sectors. Since misalignment is likely to affect all blocks on a device, increased latency and reduced performance are a common result.
By default, ASM diskgroups placed on raw devices with a 4KB block size will be aligned, but if partitioned devices are used care should be taken to ensure the partition starts on a 4KB boundary, since many operating systems do not do this by default.
Since a 4096 physical byte sector contains eight 512 byte logical sectors, one simple solution is to ensure that any partition starts on a multiple of 8 logical sectors. For example, a value of 40 sectors would allow ample room for the partition table and still result in the partition starting on a physical sector boundary.
Configuration Steps
1. Configure storage devices to present storage using 512e mode.
2. Configure multipathing software if required
3. If using partitions, create aligned partition for use with ASM. For example, in Linux:
parted <device> mklabel gpt
parted <device> -s -- mkpart primary 40s -0
parted <device> unit s print
4. If ASMLib is to be used, install and configure ASMLib and then label devices
5. Follow the standard installation steps for Oracle Database and Grid Infrastructure
6. Create database as normal (e.g. using DBCA)
7. From the database instance and at the SQLPLUS prompt set _disk_sector_size_override:
ALTER SYSTEM SET “_DISK_SECTOR_SIZE_OVERRIDE”=”TRUE”;
8. Create new online redo log files for the database:
ALTER DATABASE ADD LOGFILE GROUP 5 SIZE 100M BLOCKSIZE 4096;
9. Drop old online redo log files
REFERENCES
NOTE:1630790.1 - Supporting ASM on 4K/4096 Sector Size (SECTOR_SIZE) Disks
https://blog.****.net/techsupporter/article/details/52447528