In this section: |
All computations are done in floating-point arithmetic. Packed fields are converted to D internally, then back to P.
When a field with decimal places is computed to an integer field, the decimal places are truncated, and the resulting value is the integer part of the input value.
When a field with decimal places is computed from one format to another, two conversions take place:
The following example illustrates some differences in the way packed fields, floating-point fields, and integer fields are stored and displayed. It also shows database values redefined to a format with a larger number of decimal places.
Master File
FILE=EXAMPLE, SUFFIX=FOC SEGNAME=ONLY, SEGTYPE=S1,$ FIELD=PACKED2,,P9.2,$ FIELD=DOUBLE2,,D9.2,$ FIELD=FLOAT2,, F9.2,$ FIELD=INTEGER,,I9 ,$
Program to Load Data
This MODIFY creates a file with four fields: a P field with two decimal places, a D field with two decimal places, an F field with two decimal places, and an integer field. The same data values are then loaded into each field.
CREATE FILE EXAMPLE MODIFY FILE EXAMPLE FIXFORM PACKED2/9 X1 DOUBLE2/9 X1 FLOAT2/9 X1 INTEGER/9 MATCH PACKED ON MATCH REJECT ON NOMATCH INCLUDE DATA 1.6666666 1.6666666 1.6666666 1.6666666 125.16666 125.16666 125.16666 125.16666 5432.6666 5432.6666 5432.6666 5432.6666 4.1666666 4.1666666 4.1666666 4.1666666 5.5 5.5 5.5 5.5 106.66666 106.66666 106.66666 106.66666 7.2222222 7.2222222 7.2222222 7.2222222 END
Report Request
A DEFINE command creates temporary fields that are equal to PACKED2, DOUBLE2, and FLOAT2, with redefined formats containing four decimal places instead of two. These DEFINE fields illustrate the differences in the way packed fields and floating-point fields are stored and displayed.
The request prints the values and a total for all four database fields, and for the three DEFINE fields.
DEFINE FILE EXAMPLE PACKED4/P9.4=PACKED2; DOUBLE4/D9.4=DOUBLE2; FLOAT4/D9.4=FLOAT2; END
TABLE FILE EXAMPLE PRINT PACKED2 PACKED4 DOUBLE2 DOUBLE4 FLOAT2 FLOAT4 INTEGER ON TABLE SUMMARIZE END
The resulting report follows:
PAGE 1 PACKED2 PACKED4 DOUBLE2 DOUBLE4 FLOAT2 FLOAT4 INTEGER ------- ------- ------- ------- ------ ------ ------- 1.67 1.6700 1.67 1.6667 1.67 1.6667 2 125.17 125.1700 125.17 125.1667 125.17 125.1667 125 5432.67 5432.6700 5,432.67 5,432.6666 5432.66 5,432.6641 5433 4.17 4.1700 4.17 4.1667 4.17 4.1667 4 5.50 5.5000 5.50 5.5000 5.50 5.5000 6 106.67 106.6700 106.67 106.6667 106.67 106.6667 107 7.22 7.2200 7.22 7.2222 7.22 7.2222 7 TOTAL 5683.07 5683.0700 5,683.06 5,683.0555 5683.04 5,683.0529 5684
In this example, the PACKED2 sum is an accurate sum of the displayed values, which are the same as the stored values. The PACKED4 values and total are the same as the PACKED2 values.
The DOUBLE2 sum looks off by .01; it is not the sum of the printed values but a rounded sum of the stored values. The DEFINEd DOUBLE4 values show that the DOUBLE2 values are actually rounded from the stored values. The DOUBLE4 values and sum show more of the decimal places from which the DOUBLE2 values are rounded.
The FLOAT2 total seems off by .03. Like the DOUBLE2 total, the FLOAT2 total is a rounded total of the stored FLOAT2 values. F fields are not accurate beyond eight digits, as the FLOAT4 column shows.
The integer sum is an accurate total. Like packed fields, the storage values and displayed values are the same.
DEFINE and COMPUTE may give different results for rounded fields. DEFINE fields are treated like data source fields, while COMPUTE fields are calculated on the results of the display command in the TABLE request. The following example illustrates this difference:
DEFINE FILE EXAMPLE DEFP3/P9.3=PACKED2/4; END
TABLE FILE EXAMPLE PRINT PACKED2 DEFP3 COMPUTE COMPP3/P9.3=PACKED2/4; ON TABLE SUMMARIZE END
The following report results:
PAGE 1 PACKED2 DEFP3 COMPP3 ------- ----- ------ 1.67 .417 .417 125.17 31.292 31.292 5432.67 1358.167 1358.167 4.17 1.042 1.042 5.50 1.375 1.375 106.67 26.667 26.667 7.22 1.805 1.805 TOTAL 5683.07 1420.765 1420.767
The DEFP3 field is the result of a DEFINE. The values are treated like data source field values. The printed total, 1420.765, is the sum of the printed DEFP3 values, just as the PACKED2 total is the sum of the printed PACKED2 values.
The COMPP3 field is the result of a COMPUTE. The printed total, 1420.767, is calculated from the total sum of PACKED2 (5683.07 / 4).
Information Builders |