#Step 1 #Find and replace schemaName #Find and replace tableName #Step 2 #Find the table #Via Databricks run the Spark SQL query below #default is schema, change as needed DESC FORMATTED schemaName.tableName; #Step 3 #From the table returned scroll down to "location" and copy the field value #Find and replace locationFieldValue #Step 5 #Via Databricks using Spark SQL drop the table DROP TABLE tableName #Step 6 #Find and replace locationFieldValue #By the means you use to interact with Databricks File System (dbfs), e.g. cmd python virtual environment #Run command below dbfs rm -r "locationFieldValue"
Tag Archives: files

How to move/rename Database files in SQL Server
An example use case for the process below could be you need to move database files to a new drive. Another example might be your organisation intends to run a legacy database along side a new updated database with both sharing the same database name in the same instance with the files located in the same directory with the same names. Obviously this cannot be done and requires the database names to differ and the files to be renamed or not exist in the same directory.
For example AdventureWorks might become AdventureWorks_Legacy while a new and improved AdventureWorks database retains the original database name. The associated database file names would also need to be changed/moved to reflect this.
Someone might also want to do something like this for test purposes but obviously having test resources in a live environment would not be recommended if avoidable.
The first step to moving and renaming the files is to copy and modify the script below. Note the script below assumes you want to move and change the names of the files. To avoid any database conflicts you only need to do one or the other.
/* Find & Replace DbName with the name of the Database you are working with */ USE [DbName]; /* Changing Physical names and paths Replace 'C:\...\NewDbName.mdf' with full path of new Db file to be used */ ALTER DATABASE DbName MODIFY FILE ( NAME = ' DbName ' ,FILENAME = 'C:\...\NewDbName.mdf' ); /* Replace 'C:\...\NewDbName_log.ldf' with full path of new Db log file to be used */ ALTER DATABASE DbName MODIFY FILE ( NAME = ' DbName _log' ,FILENAME = 'C:\...\NewDbName_log.ldf' ); /* Changing logical names */ ALTER DATABASE DbName MODIFY FILE ( NAME = DbName ,NEWNAME = NewDbName ); ALTER DATABASE DbName MODIFY FILE ( NAME = DbName_log ,NEWNAME = NewDbName_log );
- Open Microsoft SQL Server Management Studio (SSMS).
- Connect to the server that houses the Db you are working with.
- Run the modified script
- Right click on the Db in SSMS and select Tasks > Take Offline
- If you are moving the database files log into the server that houses the database files and copy and move the MDF and LDF files to the location you specified in first two alter commands. If the script specifies new names rename the copied files to match the names given in the script exactly.
- Go back to SSMS and right click on the Db and select Tasks > Bring Online.
- If you have moved the files once the database is back online and confirmed working as expected the unused original files can be deleted.
- Now you can rename the Db to the new name if you wish using SSMS.
How to demonstrate the fill ratio of separate tempdb files of equal size in SQL Server
This topic reminds of me this little quiz of which jug will fill first.
Pretty much all of the documentation and recommendations out there say to keep the tempdb data files the same size so that the round-robin data flow works properly, i.e. the tempdb data files fill up evenly. This means that the data for a large temp table is actually split across the files and does not reside in one file.
Below is the code necessary to prove this scenario.
I tested this process onĀ Microsoft SQL Server 2012 – Service Pack 1.
If working with a default installation of SQL Server Express The below script should print out the code to generate four equally sized (500 Mb) tempdb data files with no auto growth.
Run the script against the instance, review the print out and then copy/paste and run it against the instance
SET NOCOUNT ON IF OBJECT_ID('tempdb..#sfs') IS NOT NULL DROP TABLE #sfs; DECLARE @TempDbDirectory VARCHAR(MAX) DECLARE @Sql VARCHAR(MAX) CREATE TABLE #sfs ( fileid TINYINT ,filegroupid TINYINT ,totalextents INT ,usedextents INT ,dbfilename SYSNAME ,physfile VARCHAR(255) ); INSERT INTO #sfs EXEC ('USE tempdb; DBCC showfilestats;'); SET @TempDbDirectory = ( SELECT REPLACE(physfile, 'tempdb.ndf', '') FROM #sfs WHERE dbfilename = 'tempdev' ) SET @Sql = ' USE [tempdb] GO DBCC SHRINKFILE ( N''tempdev'' ,100 ) GO USE [tempdb] GO ALTER DATABASE [tempdb] MODIFY FILE ( NAME = N''tempdev'' ,NEWNAME = N''tempdev1'' ) GO USE [master] GO ALTER DATABASE [tempdb] MODIFY FILE ( NAME = N''tempdev1'' ,FILENAME = N''' + @TempDbDirectory + 'tempdb.ndf'' ,SIZE = 512000 KB ,FILEGROWTH = 0 ) GO ALTER DATABASE [tempdb] ADD FILE ( NAME = N''tempdev2'' ,FILENAME = N''' + @TempDbDirectory + 'tempdb2.ndf'' ,SIZE = 512000 KB ,FILEGROWTH = 0 ) GO ALTER DATABASE [tempdb] ADD FILE ( NAME = N''tempdev3'' ,FILENAME = N''' + @TempDbDirectory + 'tempdb3.ndf'' ,SIZE = 512000 KB ,FILEGROWTH = 0 ) GO ALTER DATABASE [tempdb] ADD FILE ( NAME = N''tempdev4'' ,FILENAME = N''' + @TempDbDirectory + 'tempdb4.ndf'' ,SIZE = 512000 KB ,FILEGROWTH = 0 ) GO ALTER DATABASE [tempdb] MODIFY FILE ( NAME = N''templog'' ,FILEGROWTH = 512000 KB ) GO ' PRINT @Sql


/* Credit for this script goes to: DAVE TURPIN http://www.daveturpin.com/2011/07/how-to-drop-a-tempdb-database-file/ */ -- Is there data in the second file of tempdb? --drop table #sfs --drop table #fixed_drives --drop table #output_table --drop table #databases --drop table #dbf --drop table #fg IF OBJECT_ID('tempdb..#sfs') IS NOT NULL DROP TABLE #sfs; IF OBJECT_ID('tempdb..#fixed_drives') IS NOT NULL DROP TABLE #fixed_drives; IF OBJECT_ID('tempdb..#output_table') IS NOT NULL DROP TABLE #output_table; IF OBJECT_ID('tempdb..#databases') IS NOT NULL DROP TABLE #databases; IF OBJECT_ID('tempdb..#dbf') IS NOT NULL DROP TABLE #dbf; IF OBJECT_ID('tempdb..#fg') IS NOT NULL DROP TABLE #fg; -------------------------- -- Save result set from showfilestats -------------------------- CREATE TABLE #sfs ( fileid TINYINT ,filegroupid TINYINT ,totalextents INT ,usedextents INT ,dbfilename SYSNAME ,physfile VARCHAR(255) ); ------------------------------ -- Save result set from sys.database_files ------------------------------ CREATE TABLE #dbf ( [file_id] INT ,file_guid UNIQUEIDENTIFIER ,[type] TINYINT ,type_desc NVARCHAR(60) ,data_space_id INT ,[name] SYSNAME ,physical_name NVARCHAR(260) ,[state] TINYINT ,state_desc NVARCHAR(60) ,size INT ,max_size INT ,growth INT ,is_media_ro BIT ,is_ro BIT ,is_sparse BIT ,is_percent_growth BIT ,is_name_reserved BIT ,create_lsn NUMERIC(25, 0) ,drop_lsn NUMERIC(25, 0) ,read_only_lsn NUMERIC(25, 0) ,read_write_lsn NUMERIC(25, 0) ,diff_base_lsn NUMERIC(25, 0) ,diff_base_guid UNIQUEIDENTIFIER ,diff_base_time DATETIME ,redo_start_lsn NUMERIC(25, 0) ,redo_start_fork_guid UNIQUEIDENTIFIER ,redo_target_lsn NUMERIC(25, 0) ,redo_target_fork_guid UNIQUEIDENTIFIER ,back_lsn NUMERIC(25, 0) ); ------------------------------ -- Save result set from sys.filegroups select * from sys.filegroups ------------------------------ CREATE TABLE #fg ( [name] SYSNAME ,data_space_id INT ,[type] CHAR(2) ,type_desc NVARCHAR(60) ,is_default BIT ,is_system BIT ,[filegroup_id] UNIQUEIDENTIFIER ,log_filegroup_id INT ,is_read_only BIT ); -- Populate #disk_free_space with data CREATE TABLE #fixed_drives ( DriveLetter CHAR(1) NOT NULL ,FreeMB INT NOT NULL ); INSERT INTO #fixed_drives EXEC master..xp_fixeddrives; CREATE TABLE #output_table ( DatabaseName SYSNAME ,FG_Name SYSNAME ,GB_Allocated NUMERIC(8, 2) ,GB_Used NUMERIC(8, 2) ,GB_Available NUMERIC(8, 2) ,DBFilename SYSNAME ,PhysicalFile SYSNAME ,Free_GB_on_Drive NUMERIC(8, 2) ); SELECT NAME AS DBName INTO #databases FROM sys.databases WHERE database_id <= 4 AND state_desc = 'ONLINE'; DECLARE @dbname SYSNAME; SELECT @dbname = ( SELECT TOP (1) DBName FROM #databases ); DELETE FROM #databases WHERE DBName = @dbname; WHILE @dbname IS NOT NULL BEGIN -- Get the file group data INSERT INTO #sfs EXEC ('USE ' + @dbname + '; DBCC showfilestats;'); INSERT INTO #dbf EXEC ('USE ' + @dbname + '; SELECT * FROM sys.database_files;'); INSERT INTO #fg EXEC ('USE ' + @dbname + '; SELECT * FROM sys.filegroups;'); -- Wrap it up! INSERT INTO #output_table ( DatabaseName ,FG_Name ,GB_Allocated ,GB_Used ,GB_Available ,DBFilename ,PhysicalFile ,Free_GB_on_Drive ) SELECT @dbname AS DATABASE_NAME ,fg.NAME AS [File Group Name] ,CAST(((sfs.totalextents * 64.0) / 1024000.0) AS NUMERIC(8, 2)) AS GB_Allocated ,CAST(((sfs.usedextents * 64.0) / 1024000.0) AS NUMERIC(8, 2)) AS GB_Used ,CAST((((sfs.totalextents - sfs.usedextents) * 64.0) / 1024000.0) AS NUMERIC(8, 2)) AS GB_Available ,sfs.dbfilename ,sfs.physfile ,CAST((fd.FreeMB / 1000.0) AS NUMERIC(8, 2)) AS Free_GB_on_Drive FROM #sfs sfs INNER JOIN #dbf dbf ON dbf.[file_id] = sfs.fileid INNER JOIN #fg fg ON fg.data_space_id = sfs.filegroupid INNER JOIN #fixed_drives fd ON fd.DriveLetter = SUBSTRING(sfs.physfile, 1, 1); SELECT @dbname = ( SELECT TOP (1) DBName FROM #databases ); IF @dbname IS NOT NULL DELETE FROM #databases WHERE DBName = @dbname; TRUNCATE TABLE #sfs; TRUNCATE TABLE #dbf; TRUNCATE TABLE #fg; END SELECT CONVERT(INT, CONVERT(CHAR, current_timestamp, 112)) AS CaptureDate ,DatabaseName ,FG_Name ,GB_Allocated ,GB_Used ,GB_Available ,DBFilename ,PhysicalFile ,Free_GB_on_Drive FROM #output_table ORDER BY DatabaseName ,FG_Name
SET NOCOUNT ON; DECLARE @x INT SET @x = 1 CREATE TABLE #MyTempTable (id BIGINT) WHILE @x < 6553600 BEGIN ; INSERT INTO #MyTempTable (id) VALUES (1) SET @x = @x + 1 END;

How to run multiple SQL scripts automatically in order.
If you’ve been working towards a new deployment to a live database chances are you have written several scripts (possibly dozens) that have been developed/tested against the the development server.
Now the time has come to put the update live. Which would require executing each script against the live database.
This task can be automated by using a very handy batch script to run against the directory the files are saved in.
Caveat: This process does not take into account error handling or rollbacks, it’s just a simple example people can build on.
In order for this to work the files must have been named in a manner that the necessary order of execution corresponds to ascii sort order, i.e. 001_CreateTable.sql, 002_PopulateTable etc. This is standard practice for sql file naming conventions.
Simply create a .BAT file with the following command:
(Swap servername and databaseName for your required server and database names, TIP: SELECT @@servername can provide you with the full server name.)
for %%G in (*.sql) do sqlcmd /S servername /d databaseName -E -i"%%G" pause
Place this .BAT file in the directory from which you want the .SQL files to be executed, double click the .BAT file and the command will loop and execute every SQL script in the folder.
How to copy only files from a folder and respective subfolders using a batch script
Ok so here’s a nifty little batch script that will loop through all the directories in a directory to gather all the subsequent files into a new directory. (Code at bottom of the page)
(For those of you that don’t know what a batch script is here’s a quick definition. A batch file is a text file that contains a sequence of commands for a computer operating system. It’s called a batch file because it batches (bundles or packages) into a single file a set of commands that would otherwise have to be presented to the system interactively from a keyboard one at a time.)
Here’s a diagram explaining how the batch script provided at the bottom of the page works. (Note that the folders aren’t copied just the files.)
So why would you use this?
Lets say you’re dealing with a system that has outputted thousands of folders into a single folder and each folder contains files.
You could be looking at millions of files to be gathered, a process that cannot be done manually.
Or a simpler use case might be you’ve downloaded thousands of movies and each was put into a separate folder. Now you just want the media files in one location.
How to use:
Open the program notepad.
Copy the code below and past it into notepad saving the file with the extension .bat
Place the batch file in the parent folder, i.e. the folder all the other folders are in.
Double click the batch file.
The batch file will create a folder one directory level above where the parent folder is located and copy all the files to this location.
An example of how to use the file is as follows:
If you had the parent folder “My Movies” on your desktop.
Pasting the batch file into this folder, and double clicking on it, would copy every movie into a folder called “Copied from My Movies” on your desktop.
And here’s the code,
Adiós.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@ECHO OFF | |
TITLE WAIT ! | |
:: ASSIGN THE FILE PATH OF BATCH FILE TO A VARIABLE | |
SET "sourceDir=%CD%" | |
:: GET THE NAME OF THE FOLDER WHICH THE BATCH FILE IS IN | |
FOR %%a IN (.) DO SET currentFolder=%%~na | |
:: GO UP ONE DIRECTORY | |
CD .. | |
:: MAKE A DYNAMIC FOLDER NAME | |
::SET folderName=Copied From %currentFolder% | |
SET "folderName=Copied from %currentFolder%" | |
:: CREATE A FOLDER TO PUT THE COPIED FILES IN | |
:: IF FOLDER ALREADY EXISTS DELETE IT | |
IF EXIST "%folderName%" RMDIR "%folderName%" /S /Q | |
MKDIR "%folderName%" | |
:: ASSIGN DESTINATION FOLDER TO A VARIABLE | |
SET "destinationFolder=%CD%\%folderName%" | |
:: CREATE A LOG FILE IN DESTINATION FOLDER | |
SET "_report=%destinationFolder%\logxcopy.txt" | |
:: CREATE ERROR MESSAGE | |
IF NOT EXIST "%sourceDir%" (ECHO.Could not find %sourceDir% &GoTo:DONE) | |
:: OVERWRITE PREVIOUS LOG | |
>"%_report%" ( | |
echo.%date% – %time% | |
echo.————————————————— | |
echo. | |
) | |
:: COPY FILES | |
FOR /F "Delims=" %%! IN ('DIR "%sourceDir%\" /b /s /a-d 2^>NUL') DO ( | |
@ECHO.%%! &( | |
@XCOPY "%%!" "%destinationFolder%\" /i /y /h /f /c >>"%_report%",2>&1) | |
) | |
:DONE | |
TITLE,Done……. | |
ECHO.&PAUSE>NUL |