Written by • August 21, 2020• 13:02• Database administration, Memory Optimization

Workplace Encounters: Reclaiming Space from an Oversized Database

HomeDatabase administration, Memory OptimizationWorkplace Encounters: Reclaiming Space from an Oversized Database
CodingSight - Workplace Encounters: Reclaiming Space from an Oversized Database

INTRODUCTION

This document records steps, scripts, and ideology behind the maintenance performed on a production database between 22nd April 2015 and 23rd April 2015. The details are specific to an application, but the principles may be useful to others in need of doing similar operations in production.

BACKGROUND

The database had grown to over 1.44TB with the table EPOEvents consuming over half of this size (792GB). Efforts had been made in the past to delete entries in this table older than 90 days. Unfortunately, it proved futile. The job scheduled to achieve this could never complete during any session. The reason was the poor performance of the database.

Upon discussion, a decision was made to purge the entire EPOEvents table by truncating it. The further decision was to prepare the database correctly for the expected data growth in the future.

The details of the database server in question are below:

  
SERVERNAMESVR-EPO-02
INSTANCE NAMESVR-EPO-02\ENG_AVSERVER
OS VERSIONWindows 2008 R2 Enterprise (SP1) 64-bit
DB VERSIONMicrosoft SQL Server 2008 R2 (SP1) 64-bit
IP ADDRESSXX.XX.XX.XX
DATABASE NAMEePO4_SVR-EPO-02

EPOEVENTS TABLE

The script defining the EPOEvents table is documented in Appendix I.

It is worth mentioning that the AutoID column of this table is in Foreign Key relationships with the tables HIP8_EventInfo, HIP8_IPSEventParameter, and SCOR_EVENTS. This affected the approach used to truncate the EPOEvents table. The summary details are in the table below:

This information was extracted using the statement in Listing 1.

Listing 1: Extracting Foreign Key information for a specified table
USE [ePO4_SVR-EPO-02]
GO
EXEC sp_fkeys 'EPOEvents'

PROCEDURE SUMMARY

Summary of Procedure Used During the Change:

Deploy five 200GB Drives

Five new drives were deployed on the server. The complete list of all drives on the server is as follows:

DRIVEUSENEW?COMMENT
CSystem driveNODefault Cluster Size (4KB)
DApplication DriveNODefault Cluster Size (4KB)
IMSSQL Data (OLD)NODefault Cluster Size (4KB)
JMSSQL LogNORecommended Cluster Size for SQL (64K)
MMSSQL Data (FG_LARGE FileGroup)YESRecommended Cluster Size for SQL (64K)
NMSSQL Data (FG_LARGE FileGroup)YESRecommended Cluster Size for SQL (64K)
OMSSQL Data (FG_LARGE FileGroup)YESRecommended Cluster Size for SQL (64K)
PMSSQL Data (PRIMARY FileGroup)YESRecommended Cluster Size for SQL (64K)
QMSSQL TempDBYESRecommended Cluster Size for SQL (64K)

All new drives were formatted using 64K cluster size, as recommended by Microsoft for drives containing MS SQL Server data or log files[1]. Drive J was also reformatted after taking a backup of the database.

[1] Full discussion is accessible at https://msdn.microsoft.com/en-us/library/dd758814.aspx

Fig. 1 Default Cluster Size for Drive C:
Fig. 2 Recommended Cluster Size for Drive M:

Stop EPO Application Services

All application services were stopped to ensure no update on the database during the period of the change. Application Services stopped (or confirmed down) are as follows:

  1. McAfee ePolicy Orchestrator 4.6.8 Application Server
  2. McAfee ePolicy Orchestrator 4.6.8 Event Parser
  3. McAfee ePolicy Orchestrator 4.6.8 Server

Backup the EPO Database

The first backup of the database had been taken before any action took place on the existing structure, using the script in Listing 2.

Listing 2: First backup of ePO4_SVR-EPO-02 database

BACKUP DATABASE [ePO4_SVR-EPO-02] to
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A01.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A02.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A03.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A04.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A05.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A06.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A07.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A08.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A09.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_A10.bak'
WITH STATS = 5, COMPRESSION, NAME = N'ePO4_SVR-EPO-02 First Backup Before Purge'

Create a New Filegroup

A new file group called FG_LARGE was created with the equivalent of the script in Listing 3. All files added to the filegroup were defined with a 40GB initial size, 1G increment, and 100GB max size.

Listing 3: Creation of Filegroup FG_LARGE

USE [master]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILEGROUP [FG_LARGE]
GO

USE [master]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_01', FILENAME = N'M:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_01.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_02', FILENAME = N'M:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_02.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_03', FILENAME = N'N:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_03.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_04', FILENAME = N'N:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_04.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_05', FILENAME = N'O:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_05.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO
ALTER DATABASE [ePO4_SVR-EPO-02] ADD FILE ( NAME = N'ePO4_SVR-EPO_FG_LARGE_06', FILENAME = N'O:\MSSQL\DATA\ePO4_SVR-EPO_FG_LARGE_06.ndf' , SIZE = 41943040KB , MAXSIZE = 104857600KB , FILEGROWTH = 1048576KB ) TO FILEGROUP [FG_LARGE]
GO

Truncate the Table EPOEvents

Listing 4 shows the complete set of tasks required to truncate the EPOEvents table successfully. The script includes notes for clarity.

The fundamental issue to note is that it is impossible to truncate a table participating as parent in a Foreign Key relationship. It would violate referential integrity. Deleting would work and preserve referential integrity since Foreign Keys are typically defined with ON DELETE CASCADE or ON DELETE SET NULL clauses.

Listing 4: Truncating EPOEvents table and Relocating to New Tablespace

/****** Check Foreign Keys Referencing EPOEvents ******/

USE [ePO4_SVR-EPO-02]
GO
EXEC sp_fkeys 'EPOEvents'


/****** Script Identified Foreign Keys ******/

USE [ePO4_SVR-EPO-02]
GO

ALTER TABLE [dbo].[HIP8_IPSEVENTPARAMETER]  WITH CHECK ADD  CONSTRAINT [FK_HIP8_IPSEVENTPARAMETER_EPOEVENTS] FOREIGN KEY([EVENTID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[HIP8_IPSEVENTPARAMETER] CHECK CONSTRAINT [FK_HIP8_IPSEVENTPARAMETER_EPOEVENTS]
GO


USE [ePO4_SVR-EPO-02]
GO

ALTER TABLE [dbo].[HIP8_EVENTINFO]  WITH CHECK ADD  CONSTRAINT [FK_HIP8_EVENTINFO_EPOEVENTS] FOREIGN KEY([EVENTID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[HIP8_EVENTINFO] CHECK CONSTRAINT [FK_HIP8_EVENTINFO_EPOEVENTS]
GO

USE [ePO4_SVR-EPO-02]
GO

ALTER TABLE [dbo].[SCOR_EVENTS]  WITH CHECK ADD  CONSTRAINT [SCOR_EVENTS_EPO_EVENTS] FOREIGN KEY([EPO_EVENT_AUTO_ID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON UPDATE CASCADE
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[SCOR_EVENTS] CHECK CONSTRAINT [SCOR_EVENTS_EPO_EVENTS]
GO

/****** Backup the Child Tables ******/

USE [ePO4_SVR-EPO-02]
GO
SELECT * INTO HIP8_IPSEVENTPARAMETER_BAK FROM HIP8_IPSEVENTPARAMETER;
SELECT * INTO HIP8_EVENTINFO_BAK FROM HIP8_EVENTINFO;
SELECT * INTO SCOR_EVENTS_BAK FROM SCOR_EVENTS;
GO

/****** Truncate Child Tables ******/

USE [ePO4_SVR-EPO-02]
GO
TRUNCATE TABLE HIP8_IPSEVENTPARAMETER;
TRUNCATE TABLE HIP8_EVENTINFO;
TRUNCATE TABLE SCOR_EVENTS;

/****** Drop Foreign Keys ******/

USE [ePO4_SVR-EPO-02]
GO
ALTER TABLE HIP8_IPSEVENTPARAMETER DROP CONSTRAINT [FK_HIP8_IPSEVENTPARAMETER_EPOEVENTS];
ALTER TABLE HIP8_EVENTINFO DROP CONSTRAINT [FK_HIP8_EVENTINFO_EPOEVENTS];
ALTER TABLE SCOR_EVENTS DROP CONSTRAINT [SCOR_EVENTS_EPO_EVENTS];

/***** Truncate EPOEvents Table ******/


USE [ePO4_SVR-EPO-02]
GO
TRUNCATE TABLE EPOEVENTS;

Move EPOEvents to the New Filegroup

The EPOEvents table was moved to the FG_LARGE Filegroup using the script in Listing 5.

Listing 5: Moving EPOEvents Filegroup FG_LARGE

/***** Move EPOEvents to the New Filegroup FG_LARGE ******/
-- This is achieved by recreating the Clustered Index used by Primary Key on this table
-- First Step is to Script the Clustered Index

USE [ePO4_SVR-EPO-02]
GO
ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [PK_EPOEvents_AutoID] PRIMARY KEY CLUSTERED 
(
	[AutoID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
GO

-- Second Step is to drop and recreate the Clustered Index after changing
-- the Filegroup the Script (observe the ON Clause in the create statement below:
-- Filegroup was previously PRIMARY and is now FG_LARGE)

-- Drop Statement
USE [ePO4_SVR-EPO-02]
GO
ALTER TABLE [dbo].[EPOEvents] DROP  CONSTRAINT [PK_EPOEvents_AutoID] 

-- Create Statement
USE [ePO4_SVR-EPO-02]
GO
ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [PK_EPOEvents_AutoID] PRIMARY KEY CLUSTERED 
(
	[AutoID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [FG_LARGE]
GO

Recreate Foreign Keys on Child Tables

The Foreign Keys on Child tables identified in step 4.1.5, were recreated with the script in Listing 6.

Listing 6: Truncating EPOEvents table and Relocating to New Filegroup

USE [ePO4_SVR-EPO-02]
GO

ALTER TABLE [dbo].[HIP8_IPSEVENTPARAMETER]  WITH CHECK ADD  CONSTRAINT [FK_HIP8_IPSEVENTPARAMETER_EPOEVENTS] FOREIGN KEY([EVENTID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[HIP8_IPSEVENTPARAMETER] CHECK CONSTRAINT [FK_HIP8_IPSEVENTPARAMETER_EPOEVENTS]
GO

ALTER TABLE [dbo].[HIP8_EVENTINFO]  WITH CHECK ADD  CONSTRAINT [FK_HIP8_EVENTINFO_EPOEVENTS] FOREIGN KEY([EVENTID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[HIP8_EVENTINFO] CHECK CONSTRAINT [FK_HIP8_EVENTINFO_EPOEVENTS]
GO

ALTER TABLE [dbo].[SCOR_EVENTS]  WITH CHECK ADD  CONSTRAINT [SCOR_EVENTS_EPO_EVENTS] FOREIGN KEY([EPO_EVENT_AUTO_ID])
REFERENCES [dbo].[EPOEVENTS] ([AUTOID])
ON UPDATE CASCADE
ON DELETE CASCADE
GO

ALTER TABLE [dbo].[SCOR_EVENTS] CHECK CONSTRAINT [SCOR_EVENTS_EPO_EVENTS]
GO

Script the EPOEvents Table Definition

As a backup, it was ideal for scripting the extract of the DDL of the EPOEvents table. This was done by right-clicking and selecting the option Script Table as > CREATE To > New Query Editor Window from the SSMS[1] Object Explorer. The resulting script can be saved to a .sql file.

Fig. 4 Scripting the EPOEvents Table

Backup the EPO Database

A second backup of the EPO database was taken with the script in Listing 7 to prepare for the next stage (datafile shrinking).

Listing 7: Second backup of EPO Database

BACKUP DATABASE [ePO4_SVR-EPO-02] TO
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C01.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C02.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C03.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C04.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C05.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C06.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C07.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C08.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C09.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_C10.bak'
WITH STATS = 5, COMPRESSION, NAME = N'ePO4_SVR-EPO-02 Backup After Purge'

Shrink Datafiles in the PRIMARY Filegroup

Shrinking is necessary to reclaim the space freed up by truncating the EPOEvents table. It was also crucial in this case, as the data files required relocation to a smaller drive.

The data files were relocated from the drive I – a 3 TB drive to drive P, a 200 GB drive.

The three data files in the PRIMARY Filegroup were logically named ePO4_SVR-EPO-01 (~300GB), ePO4_SVR-EPO-02 (~500GB), and ePO4_SVR-EPO-03(~400GB).

The target was to shrink all files to 40000 MB each. The task demanded five to seven iterations, each removed 50000 MB.

Listing 8: Shrinking Datafiles in the PRIMARY Filegroup

USE [ePO4_SVR-EPO-02]
GO
DBCC SHRINKFILE (N'ePO4_SVR-EPO-01' , 40000)
DBCC SHRINKFILE (N'ePO4_SVR-EPO-02' , 40000)
DBCC SHRINKFILE (N'ePO4_SVR-EPO-03' , 40000)
GO

Backup the EPO Database

A second backup of the EPO database was taken using the script in Listing 5 to prepare for the next stage (dropping the existing database). The script is present in Listing 9.

Listing 9: First backup of ePO4_SVR-EPO-02 database

BACKUP DATABASE [ePO4_SVR-EPO-02] TO
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E01.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E02.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E03.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E04.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E05.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E06.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E07.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E08.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E09.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E10.bak'
WITH STATS = 5, COMPRESSION, NAME = N'ePO4_SVR-EPO-02 Backup After Shrink'

Drop the EPO Database

The database was dropped using the SSMS GUI – right-click the database and select the “delete” option from the drop-down menu. The equivalent SQL is in Listing 10.

Listing 10: Drop the EPO Database

USE [master]
GO
DROP DATABASE [ePO4_SVR-EPO-02]
GO

Format Drive J

This step formats the drive J, using the 64K cluster size, as recommended by Microsoft for optimal I/O performance. This task was previously performed on the new drives deployed in step 4.1.1.

Figure 3 shows the option selected in the Format… dialog box.

Fig. 4 Formatting Drive J Using 64K Cluster Size

Restore the EPO Database with MOVE

The restore operation was necessary for three reasons:

  1. To format drive J containing the transaction Log Files (as above).
  2. To remove any fragmentation caused by the shrink operation.
  3. To relocate the database to new drives.

The script in Listing 11 serves for restoring. Note that the backup set used for this restore is the last backup taken in step 4.1.8. Also, observe that the MOVE option serves to relocate the data files in the PRIMARY Filegroup to drive P.

Listing 11: Drop the EPO Database

RESTORE DATABASE [ePO4_SVR-EPO-02] FROM
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E01.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E02.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E03.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E04.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E05.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E06.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E07.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E08.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E09.bak',
DISK = 'I:\MSSQL\BACKUP\ePO4_SVR-EPO-02_22Apr2015_E10.bak'
WITH STATS = 5,
MOVE 'ePO4_SVR-EPO-02' TO 'P:\MSSQL\DATA\ePO4_SVR-EPO-02.mdf',
MOVE 'ePO4_SVR-EPO-01' TO 'P:\MSSQL\DATA\ePO4_SVR-EPO-01.ndf',
MOVE 'ePO4_SVR-EPO-03' TO 'P:\MSSQL\DATA\ePO4_SVR-EPO-03.ndf'

Relocate TempDB to Drive Q

For optimal performance of large databases, it is also recommended to locate TempDB files on a dedicated drive.

Using the script in Listing 9, TempDB was relocated to drive Q. Note that the redundant TempDB data files were removed during this operation. The number of TempDB data files should match the number of physical CPUs available to SQL Server.

Listing 12: Relocate TempDB

-- Drop unnecessary TempDB files
-- The following script was run several times and required several restarts of the -- SQL instance

USE [tempdb]
GO
ALTER DATABASE [tempdb]  REMOVE FILE [tempdev_xx]
GO


-- Relocate TempDB files

USE master;
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = tempdev, FILENAME = 'Q:\MSSQL\DATA\tempdev.mdf');
GO
ALTER DATABASE tempdb 
MODIFY FILE (NAME = tempdev_01, FILENAME = 'Q:\MSSQL\DATA\tempdev_01.mdf');
GO
ALTER DATABASE  tempdb 
MODIFY FILE (NAME = templog, FILENAME = 'J:\MSSQL\LOG\templog.ldf');
GO

-- Restart SQL Server
-- Confirm the location of TempDB files

SELECT name as 'File Name', physical_name as 'File Directory'
FROM sys.master_files
WHERE database_id = DB_ID('tempdb');
GO

Start EPO Application Services

All application services were started once the SQL Server instance was confirmed OK. Application Services started are as follows:

  1. McAfee ePolicy Orchestrator 4.6.8 Application Server
  2. McAfee ePolicy Orchestrator 4.6.8 Event Parser
  3. McAfee ePolicy Orchestrator 4.6.8 Server

Confirm Tables Are Populated

The script is listing 13 confirms that the table EPOEvents and the child tables are being populated after the entire process is complete.

Listing 13: Relocate TempDB

USE [ePO4_SVR-EPO-02]
GO
SELECT COUNT(*) EPOEvents_CNT FROM EPOEvents;
SELECT COUNT(*) HIP8_EventInfo_CNT FROM HIP8_EventInfo;
SELECT COUNT(*) HIP8_IPSEventParameter_CNT FROM HIP8_IPSEventParameter;
SELECT COUNT(*) SCOR_EVENTS_CNT FROM SCOR_EVENTS;

POST CHANGE TASKS

Summary of Post Change Tasks

1          The job Custom_Purge EPO Events was created to keep the number of events captured within 100 days. It is crucial to ensure this job is always successful.

2          Excessive space previously allocated to the EPO database server can be reclaimed, particularly the drive I – currently 1 TB.

3          A backup job Custom_Daily_Backup was created during the change. Backups made by this job will drop backup sets to I:/MSSQL/Backup/. It is essential to change this path if the drive I will be removed from the system. It is also necessary to preserve these backups according to Group Backup Policy.

APPENDICES

APPENDIX I

EPOEVENTS TABLE DDL

USE [ePO4_SVR-EPO-02]
GO

/****** Object:  Table [dbo].[EPOEvents]    Script Date: 04/23/2015 01:40:46 ******/
SET ANSI_NULLS ON
GO

SET QUOTED_IDENTIFIER ON
GO

SET ANSI_PADDING ON
GO

CREATE TABLE [dbo].[EPOEvents](
	[AutoID] [int] IDENTITY(1,1) NOT NULL,
	[AutoGUID] [uniqueidentifier] NOT NULL,
	[ServerID] [nvarchar](16) NOT NULL,
	[ReceivedUTC] [datetime] NOT NULL,
	[DetectedUTC] [datetime] NOT NULL,
	[AgentGUID] [uniqueidentifier] NOT NULL,
	[Analyzer] [nvarchar](16) NOT NULL,
	[AnalyzerName] [nvarchar](64) NOT NULL,
	[AnalyzerVersion] [nvarchar](20) NOT NULL,
	[AnalyzerHostName] [nvarchar](128) NULL,
	[AnalyzerIPV4] [int] NULL,
	[AnalyzerIPV6] [binary](16) NULL,
	[AnalyzerMAC] [nvarchar](16) NULL,
	[AnalyzerDATVersion] [nvarchar](20) NULL,
	[AnalyzerEngineVersion] [nvarchar](20) NULL,
	[AnalyzerDetectionMethod] [nvarchar](128) NULL,
	[SourceHostName] [nvarchar](266) NULL,
	[SourceIPV4] [int] NULL,
	[SourceIPV6] [binary](16) NULL,
	[SourceMAC] [nvarchar](16) NULL,
	[SourceUserName] [nvarchar](128) NULL,
	[SourceProcessName] [nvarchar](128) NULL,
	[SourceURL] [nvarchar](1024) NULL,
	[TargetHostName] [nvarchar](266) NULL,
	[TargetIPV4] [int] NULL,
	[TargetIPV6] [binary](16) NULL,
	[TargetMAC] [nvarchar](16) NULL,
	[TargetUserName] [nvarchar](128) NULL,
	[TargetPort] [int] NULL,
	[TargetProtocol] [nvarchar](16) NULL,
	[TargetProcessName] [nvarchar](128) NULL,
	[TargetFileName] [nvarchar](266) NULL,
	[ThreatCategory] [nvarchar](128) NOT NULL,
	[ThreatEventID] [int] NOT NULL,
	[ThreatSeverity] [tinyint] NOT NULL,
	[ThreatName] [nvarchar](128) NOT NULL,
	[ThreatType] [nvarchar](32) NOT NULL,
	[ThreatActionTaken] [nvarchar](24) NOT NULL,
	[ThreatHandled] [bit] NULL,
	[TheTimestamp] [timestamp] NOT NULL,
 CONSTRAINT [PK_EPOEvents_AutoID] PRIMARY KEY CLUSTERED 
(
	[AutoID] ASC
)WITH (PAD_INDEX  = OFF, STATISTICS_NORECOMPUTE  = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS  = ON, ALLOW_PAGE_LOCKS  = ON) ON [PRIMARY]
) ON [PRIMARY]

GO

SET ANSI_PADDING OFF
GO

ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [DF_EPOEvents_EventID]  DEFAULT (newid()) FOR [AutoGUID]
GO

ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [DF_EPOEvents_ServerID]  DEFAULT (N'SVR-EPO-02') FOR [ServerID]
GO

ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [DF_EPOEvents_ReceivedUTC]  DEFAULT (getutcdate()) FOR [ReceivedUTC]
GO

ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [DF_EPOEvents_ThreatSeverity]  DEFAULT ((1)) FOR [ThreatSeverity]
GO

ALTER TABLE [dbo].[EPOEvents] ADD  CONSTRAINT [DF_EPOEvents_ThreatActionTaken]  DEFAULT ('none') FOR [ThreatActionTaken]
GO

APPENDIX II

JOB CUSTOM_PURGE EPO EVENTS

USE [msdb]
GO

/****** Object:  Job [Custom_Purge EPO Events]    Script Date: 04/25/2015 19:47:51 ******/
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
/****** Object:  JobCategory [[Uncategorized (Local)]]]    Script Date: 04/25/2015 19:47:51 ******/
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback

END

DECLARE @jobId BINARY(16)
EXEC @ReturnCode =  msdb.dbo.sp_add_job @job_name=N'Custom_Purge EPO Events', 
		@enabled=1, 
		@notify_level_eventlog=0, 
		@notify_level_email=0, 
		@notify_level_netsend=0, 
		@notify_level_page=0, 
		@delete_level=0, 
		@description=N'No description available.', 
		@category_name=N'[Uncategorized (Local)]', 
		@owner_login_name=N'sa', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object:  Step [Step 1]    Script Date: 04/25/2015 19:47:51 ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Step 1', 
		@step_id=1, 
		@cmdexec_success_code=0, 
		@on_success_action=1, 
		@on_success_step_id=0, 
		@on_fail_action=2, 
		@on_fail_step_id=0, 
		@retry_attempts=0, 
		@retry_interval=0, 
		@os_run_priority=0, @subsystem=N'TSQL', 
		@command=N'DECLARE @NbLignes int ; 
SELECT @NbLignes = COUNT(*) FROM EPOEvents WHERE DATEDIFF(day, DetectedUTC, GETDATE()) > ''100''; 

PRINT ''Total number of lines to delete:''; 
PRINT @NbLignes; 

WHILE (@NbLignes > 0) 
BEGIN 

        PRINT ''Total number of lines LEFT to delete: :''; 
        PRINT @NbLignes; 
        PRINT ''Start deleting 50000 lines'';
        BEGIN TRAN DEL_SET;
        DELETE TOP(500000) FROM EPOEvents WHERE DATEDIFF(day, DetectedUTC, GETDATE()) > ''100''; 
        COMMIT;
        SET @NbLignes = @NbLignes - ''50000''; 
        WAITFOR DELAY ''00:05'';
END 
PRINT ''End of Maintenance'' ; 
', 
		@database_name=N'ePO4_SVR-EPO-02', 
		@output_file_name=N'D:\MSSQL\JOBLOG\purgeepoevents.txt', 
		@flags=2
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'Purge Schedule', 
		@enabled=1, 
		@freq_type=4, 
		@freq_interval=1, 
		@freq_subday_type=1, 
		@freq_subday_interval=0, 
		@freq_relative_interval=1, 
		@freq_recurrence_factor=0, 
		@active_start_date=20130312, 
		@active_end_date=99991231, 
		@active_start_time=200000, 
		@active_end_time=235959, 
		@schedule_uid=N'3d0b8826-691d-4d3c-aca9-00cb5ce246b2'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
    IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:

GO
Tags: , , , Last modified: September 20, 2021

About the Author /

Kenneth Igiri is an Enterprise Architect with eProcess International S.A., Ecobank Group's Shared Services Centre. Kenneth has over fifteen years' experience in Information Technology with nine of those years focused on SQL Server and Oracle databases. His interests include Database Performance, HADR, Cloud Computing, Data Architecture and Enterprise Architecture. Asides from work, Kenneth teaches at Children's Church, writes faith-based fiction and helps small businesses grow. You can connect with Kenneth via his blog https://kennethigiri.com, LinkedIn, or on Amazon.com.

← CodingSight - Using SQL Server 2016 Upgrade Advisor Previous Story
Using SQL Server 2016 Upgrade Advisor
→ Next Story
SQL Server System Databases – Basic Concepts

Related Posts

SQL Percentage Calculation

How to Get SQL Percentage Calculation as Easy as Pie 

March 14, 2024• SQL Server, Work with data

Percentage calculation in SQL can be a bit like traversing an unfamiliar labyrinth. If you’re a beginner, it’s easy to get lost in a...
Read More →

Best FREE SQL Server Tools for Database Developers and Administrators!

December 07, 2023• Tools & technologies • One Comment

This article has recently been extensively revised. In particular, we reworked the article structure, and navigation, and updated...
Read More →

dbForge Edge: Your Best Universal Tool for Multidatabase Challenges

June 06, 2023• MySQL, Oracle, PostgreSQL, SQL Server, Tools & technologies

Handling several database management systems simultaneously is inevitable in working on multiple projects. At the same time, each system...
Read More →

31 Must-Know SQL Server DBA Interview Questions – Are You Prepared?

November 01, 2022• SQL Server

This article will help you to prepare for SQL Server interviews. I have explained various popular SQL Server interview questions and...
Read More →

Comments are closed.

↑
Close