Copyright: (c) 1998 - DataFlux Corporation


Introduction To FileDuster

FileDuster is a powerful tool that is a safe, easy way to eliminate unwanted files and junk that collects on your hard drives or network drives. Simply start the application and it will detect all of your connected drives. Then, choose a report to run and all desired drives will be scanned and the requested report generated. It's that simple!

There are three major reports that can be run with FileDuster.

Duplicate File Report

FileDuster will search all directories and report all discovered duplicate files spanning all connected drives – even across a network.

Size Query Report

FileDuster will report all files in all connected drives that are larger than the size requested by the User, helpful in finding and removing large, forgotten files.

Zero-Length File Report

FileDuster will find and report all files that have a length of zero to aid in the uncluttering of directories.


Drives

This dialog screen allows the disabling of any connected drives so only desired drives will be scanned. If a drive is disabled, it simply disappears from the list of drives (it is still available in other applications). Only drives that are listed will be scanned during any of the reports.

Screen Items:

OK – Returns control to the main screen, updating the drive list.

Detect – Restores the list of drives to all available, connected drives.

Disable – Causes the selected drive to be removed from the list of drives. This means that FileDuster will bypass this drive when generating its reports.

Help – Displays this help page.

Cancel – Returns control to the main screen without updating the drive list.

Note: All drives can not be disabled, at least one must appear in the list.


Size

This dialog screen will appear prior to a size report being performed. It allows the size of the file to be entered, either in kilobytes or megabytes, that a file must be in order to be listed in the report. For example, if 5 megabytes is entered, all connected drives will be scanned to find files larger than 5 megabytes, and any found will be reported.

Screen Items:

Show Size Filter As – Toggles between megabytes and kilobytes when entering size filters.

Go – Begins the size filter report.

Help – Displays this help page.

Cancel – Returns control to the main screen without running the size report.


Audit Report

The audit report screen allows viewing the history of all files that have been deleted with FileDuster (every time a file is chosen for deletion, an entry is recorded in the audit report). It can also be used to view other reports (perhaps audit reports are archived monthly) or other text files for that matter.

A file called audit.txt is created and accessed within the FileDuster directory that serves as the history log of all files deleted. This allows the retracing of steps in case a file is accidentally deleted by the user and needs to be replaced. Also remember that FileDuster places deleted files in to the recycle bin (only if the file originated on your own machine), and can be restored to their original place from the recycle bin before they are physically deleted.

Screen Items:

Open – Loads the contents of the audit report from the file onto the screen so its contents can be examined.

Print – Sends the contents of the audit report to a connected printer.

Help – Displays this help page.

Search – Searches the contents of the audit report that is displayed for the string specified in the Search String field.

Search String – String that will be searched for in the displayed audit report when the Search button is clicked.

Filename – Pathname of the file that will be loaded to the display when the Open button is clicked.

Browse – Allows use of the Windows file dialog screen to graphically search for a file.


Main Screen

This screen is where the FileDuster application is controlled and the various reports are generated.

Screen Items:

Drives – Allows choice of which drives will be scanned when running one of FileDuster's reports.

Delete – After a report is run, selected filenames may be deleted using this button.

Print – Sends the contents of a report to a connected printer.

Dupes – Generates a report of all duplicate filenames that are found across all drives displayed in the drives list.

Size – Generates a report of all files that are larger than the specified size.

Zero – Generates a report of all files that are of length zero.

Help – Displays this help page.

Contact – Information for contacting DataFlux.

About – Information about the FileDuster application.

FAQs – A list of frequently asked questions about FileDuster and their answers.

Mask – Filter that allows setting the type of file that will be considered when running a FileDuster report. For example, if the *.exe mask is selected, only files with the .exe file extension will be considered when generating the report.

Drives Field – List of all drives that will be scanned when generating a report.

www.dataflux.com – Clicking this hyperlink will launch your browser and take you to the DataFlux homepage.


Duplicate File Report

The duplicate file report will scan all directories in all listed drives to find and group all files that have the same name. Sometimes, this will be a different file with the same name, and sometimes they will be the same file, or different versions of the same file. This report identifies all duplicate files existing across all connected drives, allowing further investigation to see if a listed duplicate file warrants deletion.


Size Report

The size report scans all directories in all connected drives for files over a specified size. Often, very large files are left behind or forgotten in directories that eat up a lot of space. This report identifies them to see if they warrant deletion.

For example, if the specified size filter is 5 megabytes, all directories in all connected drives will be scanned, and all files that are larger than 5 megabytes in size will be reported.


Zero-length Report

The zero-length report will scan all directories in all connected drives for empty files or files of length zero. While these files do not take up any space, a large number of them can clutter directories and cause other files to be retrieved more slowly because they clutter directory tables.

Be sure to check whether a zero-length file is needed by another program before you delete it.


Contacting DataFlux

Address:

DataFlux Corporation

2054 Kildaire Farm Road

Suite #148

Cary, NC  27511

USA

Telephone:

(919) 461-1734

E-Mail:

info@dataflux.com – for general info

services@dataflux.com – for technical help

Internet:

http://www.dataflux.com


Frequently Asked Questions

How do I find duplicate files on my hard drive?

What is a mask?

How do I select which drives I want to dust?

How do I eliminate unnecessary large files?

Can I view a history of all deleted files?

How do I dust across a network?

Can I archive and reset the audit report?

How do I get rid of zero-length files?

Can I print my reports?

How do I know whether or not I should delete a file?

How can I clean the data within a database?

Where can I get more information?

How can I request a new feature?


FAQ - More Info

For more information about DataFlux and its line of data quality products, contact the website at http://www.dataflux.com. Or, send an e-mail to info@dataflux.com.


FAQ - Duplicate Files
FileDuster will dust all connected drives to find any files that share the same name. Simply click the Dupes button on the toolbar of the main screen begin the search. To check only for files of a certain type, change the value of the mask field on the toolbar.


FAQ - Mask
A Mask serves as a filter that allows only those files that pass the filter test to be considered during the generation of one of FileDuster's reports. For example, a "*.doc" mask will cause only files with the .doc extension to be considered while a report is being generated.


FAQ - Drive Selection
Click the Drives button on the toolbar to see a list of all connected drives. You can disable those drives that you do not want to scan while generating your desired report. At the main screen, you can see a list of all currently connected drives. One drive must remain connected at all times.


FAQ - Large Files
Click the Size button on the toolbar. Here file sizes can be selected (either in megabytes or kilobytes) that will be listed in the size report. All files that are larger than the selected size will be reported. These files can be analyzed to determine if they are worth keeping.


FAQ - Audit Report
In the Reports menu, select the Audit Report item. The audit report screen will appear. To view the current audit report, click the Open button. This will cause the audit report to appear in the window where it can be examined. Notice the filename that appears. This is the pathname of the current audit report. This can be changed to view other archived audit reports (or other text files).


FAQ - Zero-length

Zero-length files are typically harmless but can needlessly clutter up file directories. To see a report of all files that are empty (have a byte size of zero), click the Zero button. The report will appear in the main window.


FAQ - Network Drives

When FileDuster first appears, it scans all the drives that you are connected to, including any network drives. These are listed in the Drives field that appears in the toolbar. Clicking the Drives button will cause a dialog containing a list of valid drives to appear. Here, it can be determined which to scan and which to bypass during the generation of a report.


FAQ - Archive

The audit report is merely a text file called audit.txt that resides in the same directory as the FileDuster application. It can be moved to another directory for safekeeping (at which point its name may also be changed). When FileDuster initializes itself, it creates a new audit report file if the old one is no longer there.


FAQ - Print

After a report is generated, it may be easier to follow up with if a hard copy is available, especially if the report has become large. To send the contents of a report to your connected printer, simply click the Print button in the FileDuster application. After choosing your printing options, a paginated report will be sent to your printer.


FAQ - Delete

After a report is generated, it may be easier to follow up with if a hard copy is available, especially if the report has become large. To send the contents of a report to your connected printer, simply click the Print button in the FileDuster application. After choosing your printing options, a paginated report will be sent to your printer.


FAQ - Clean Data

Data within a database can be cleaned using DataFlux's more powerful database quality management tools. To first determine if you even have a data quality problem within your database - download the diagnostic tools available at no cost from DataFlux at http://www.dataflux.com. If you find you have a problem that's worth worrying about - give us a call. That's what we're here for.


FAQ - New Features

If you have an idea for a new feature you would like to see in FileDuster, email it to services@dataflux.com. We'll do our best to get it in there.


Data Quality Control

DataFlux is a Data Quality Control(sm) software company focused on the development of innovative database software products designed to significantly improve the accuracy and usability of an organization's data. Specializing in data quality control on the Windows NT platform and exploiting ODBC technology, DataFlux has quickly and effectively resolved some of the toughest data quality challenges experienced by many leading-edge small and middle market companies, as well as a number of Fortune 1,000 companies at the business unit level.

Click on a link below to find more about specific data quality control problems that our technologies can assist you in overcoming:

Database Consolidation/Merging

Unstandardized Data

Duplicate/Redundant Data

Geographical Information Auditing

Rule-based Database Auditing

Data Appearance

Dates and Year 2000


Database Consolidation/Merging

One of the most powerful and effective ways to utilize and harness the power of data is by combining it from different sources to gain a larger view of an individual, organization, or entity.

For example, imagine XYZ Insurance wants to target companies within the state of Illinois to sell health benefits policies to. Also suppose XYZ Insurance has the following databases available to it.

1 - Its own customer database, listing all companies that currently have policies with XYZ Insurance.

2 - An Illinois business demographics database, containing information such as SIC code, number of employees, annual revenue, etc.

3 - A state-registered policy database, listing all companies within the state of Illinois, who their current coverage provider is, and when those policies are due to expire.

4 - An executive database, listing all of the senior decision-makers within the organization.

1 - Customer Database

----------------------

Company Name

Address

.

.

Premium

Expiration Date

Primary Contact

2 - Illinois Demographics Database

-----------------------------------

Company Name

Address

.

.

Number of Employees

Annual Revenues

3 - Current Provider

--------------------

Company Name

Address

.

.

Current Provider

Expiration Date

4 - Key Executives

------------------

Company Name

Address

.

.

Chief Executive Officer

Purchasing Director

Chief Operating Officer

So, if XYZ Insurance now wants to ask the following question:

"What companies in the state of Illinois are not currently a customer of XYZ Insurance, have more than 200 employees, have policies that expire in the next 90 days, and who are their key decision makers?"

They can't because this information is spread across four databases. The data needs to be consolidated into one, larger view database that pulls the pertinent information from each of the four data sources. So, the final database layout would be as follows:

Prospect Targeting Database

---------------------------

Company Name

Address

Customer of XYZ(YES or NO)?

Number of Employees

Current Provider

Expiration Date

Chief Executive Officer

Purchasing Director

Chief Operating Officer

Sounds easy enough, right? Well here's the problem: The data in the common fields (Company Name, Address) of each of the databases doesn't look the same, making the records impossible to match up.

For example, IBM is 'IBM' in one database, 'I.B.M.' in the second database, 'International Business Machines' in the third, and 'ibm' in the fourth.

What's worse, the addresses are varied as well: '125 E. 9th Ave', '125 East Ninth', '125 East 9th Avenue' etc..

Utilizing DataFlux technology, these obstacles can be overcome so the larger-view database can be built and queried. Now, XYZ Insurance can ask the kind of question previously mentioned, so companies and their contacts that fit the profile can now be sent as target companies to the various field representatives in the same region.

This same example holds true when trying to gain larger prospect profiles of people as well, as the name of a person can appear in many different shapes and variations across several databases.

If your organization can benefit significantly from creating larger views of customers, prospects, companies, etc., then contact DataFlux today. We would like to take the first steps in seeing how can we help you.


Unstandardized Data

Surprisingly, the term 'unstandardized data' brings a blank stare to the face of many executives, especially those who rarely spend time in the trenches of their IT shop fighting the daily data battle.

To provide a simple example, a database may contain 'Washington Heights' represented many ways within the city field of the database:

   Washington Hts

   Washington Hts.

   Wash Hgts.

   Washington Heights

   Washngtn Heights

etc. - not to mention all the typos that could be present.

If the database is then queried for all 'Customers from Washington Heights' - obviously the results returned from the database will be substantially inaccurate. While this example is simple, it represents a common problem with many information systems.

The value of high quality, standardized data is indisputable - and it is the most important aspect of any information system.

Another Example: Organization Names

One of most common data quality contaminants encountered by an IT shop is that of varying representations for the name of an organization. For example, the following variations for 'Ford Motor Company' were found within the company name field in the customer information system of one of our clients:

   Ford

   Ford Comp

   Ford Motor Comp

   Ford Moter Company

   FORD

   Ford Mortor Company

   ford motor company

   Ford Inc.

   Ford Mortor

   Ford Motors

   Ford Motor Compnay

   FORD INC

   Ford Inc

   Ford Motor Co.

   Ford Moter Co.

   Ford Motor Co

   FORD MOTOR CO.

   Ford Plant

   Ford Motor/Waltonhill Plant

And this was for a small table of about 20,000 records - imagine the type of permutations there would be in a very large table where the number of customer records is in the millions. While data validation mechanisms may work for a state field (such as requiring the user to use 2 letters for a state), it is impossible to implement for an organization name. Also, there is no limit to the number of company name variations that prospective customers may enter in forms on a company web site. On the Internet, control over surfer data entry is very limited.

What kind of problems does this cause? It makes customer analysis by company extremely difficult.

Suppose an organization wanted to identify companies where they have had much success to target for group discounts to get to other employees within the same company. Using their customer information system to extract and analyze data by company name would be daunting with unstandardized company information - if not a futile effort entirely.

Other similar problem situations may include reporting costs by vendor, total sales by company, etc. - the possibilities are endless.

One solution is to utilize DataFlux to solve the problem. We have a diagnostic tool that we provide at no cost to allow you to gauge the extent of unstandardized data within any one of your databases. In a matter of minutes, you can snap it on to your database and begin generating unstandardized data reports. Also, if you don't want to get your hands dirty - DataFlux's world class data quality engineers can clean your data for you, and deliver it clean & pristine, as per your specifications. Give us a call or visit our web site, and begin to ease the data quality burden.


Duplicate/Redundant Data

"Are You Affected By Redundant Data?"

I am sure in your organization, the question "How can we use our data more effectively?" has often been asked. If it hasn't, it is safe to assume that you probably don't HAVE any data to begin with.

Here are some questions that may be similar to those asked in your world:

"Mr John S. Matthews of '517 E Broadway St' is applying for a loan, our database shows that a Jean Matthews of '517 East Broadway Street' has defaulted on two loans previously. Should we extend John a loan??"

"Currently, 19% of the residents of Stonebrook Village are our clients. We have just purchased a marketing list of ALL the residents of Stonebrook Village. We know that names and addresses won't be exactly the same in both databases. How we can avoid looking disorganized by marketing to existing clients?"

"We know that clients who have purchased policy A and policy C from us are ideal prospects for policy D. Unfortunately the data describing A policy holders and C policy holders does not always look the same (there are variations in the name and address fields). How can we identify which families have policy A & C but not policy D?"

"We have a large prospect database and some of our salespeople are calling on the same prospects multiple times, which is a waste of both parties' time, as well as making us look foolish - what can we do?"

"We are thinking of acquiring XYZ corporation because their product is similar to ours and we want to increase our market share. The company is being appraised with the 57,000 customers that they boast. How can we be sure that the same customers are not in their database multiple times because of name variations, multiple accounts under the same name, etc.? It would be horrifying for us to discover after acquiring the company that redundancy within the database led us to believe there were 57,000 customers when in actuality the number was closer to 40,000."

All of these questions are based on the theme of redundant data. Here are some examples of data redundancy:

names:

  Jim Johnson, James P. Johnson, Mr. James Johnson, J.P. Johnson

addresses:

  125 East Broad Boulevard, 125 E Broad Blvd, 125 E. Broad Blvd.

companies:

  International Business Machines, IBM, I.B.M., ibm

To address issue this within your organization, first ask yourself where within your databases this problem might exist - and if so what is the potential impact? If the answer is some insignificant postage costs - then you may not need to invest the time to examine the issue. However, if the impact is significant - you must first profile the types of problems that may occur. You might be asking yourself, "How can I know for sure if there is data redundancy within my database?" If the answer is that there is no way of knowing, then contact DataFlux to receive our data redundancy diagnostic tool at no cost, and find out. Or, if you prefer to have the job handled and solved by experts, then give us a call as well - our engineers are waiting to help you.


Geographical Information Auditing

Geographical information auditing can be a very important aspect of data quality. In many cases, correctness of one field can be determined by the value of another. Some examples include:

If the city and state field are listed, then correctness of the zip code can be checked.

Area codes within phone numbers can be checked against city and state fields.

Addresses can be validated and corrected against a third-party geographic information database.

County information can be verified and spell-checked.

Any time the correctness of one field can be determined by another in terms of geographical information, DataFlux can do the job!


Rule-based Data Auditing

Rule-based data auditing is the art of applying value constraints to a database. Sometimes, for example, ranges can be set up as part of the database schema. One might enter a range constraint as a salary field never exceeding $100,000 dollars. However, it becomes a little more complex if the 'rules' of the database span across several fields. Here is an example of rules that may be applied to a database:

If <Job Title> = 'Management' then <Salary> <= $75,000

If <Procedure> = 'Hysterectomy' then <Gender> = 'Female'

<Hours_Worked> <= 8

If <Contact Type> = 'Customer' and <Standing> = 'Good' then <Days_Since_Last_Contact> <= 180

Range <Discount_Percent> (0-20)

Then, the series of rules can be periodically applied to the database, and all records that do not fall within the constraints can be reported in an exceptions report so the cause can be investigated. Contact DataFlux for more information about Rule-based Data Auditing.


Data Appearance

In many situations, data appearance is very important. Whether it is consistency in the way it appears on the screen, consistency in printed form, increasing the value of data for re-sale, or trying to consolidate data in different formats - the appearance of data is of primary importance.

For example, telephone numbers can come in various formats:

(919) 555-1212

 919-555-1212

 1-919-555-1212

 1.919.555.1212

 9195551212

 ph. 919-555-1212

 919-555-1212 fax

 (919)555-1212 ext. 7901

 919.555.1212, #7901

Turning data such as this in to a uniform format can be very advantageous from a data appearance standpoint. Also, it can be important if different databases are being combined based on a field such as telephone, or if it is important that data from different sources looks the same within the target database.

Other types of data where data appearance is of critical importance is dates, social security numbers, product codes, serial numbers, currencies, etc.

Contact DataFlux today, and have your data looking good!


Dates and Year 2000

Many of the headlines and available solutions concerning the various problems with the year 2000 have dealt with source code within an application. Another large part of the problem involves the date conversion that must take place. Dates can appear within a database in many formats:

5/29/70

5-29-70

700529

5/29/1970

05/29/1970

05-29-70

etc.

So not only is this bad for appearance of data, using date as matching criteria, or as a basis for joining multiple databases, it can also contribute significantly to the coming year 2000 data scourge.

Contact DataFlux today to see how we can assist in your date conversion tasks!