Frames per Second in Falcon 4 Original & Ver.1.03
Summary of Findings, Notes & A Caution
ARHarris 2/09/99
Why not? For three good reasons: (1) the sample is not random, (2) important variables within the sample frequently suffer from being highly correlated with other variables in the sample, and (3) there are potential errors in the generating and reporting of data back to Papa Doc.
Considering (3) first, however careful the benchmark conditions generated by Papa Doc and his scrutiny of incoming observations, there is still plenty of room for error in the individual generation and accurate reporting of data. Was the system rebooted or “flushed” between tests? Were blackout and clouds really off set to “off”? Did the person accidentally or intentionally misrepresent the system producing the results? (In a large enough and true random sample (1), we could more easily assume that the errors in (3) will “wash each other out.”)
Considering (2) next, in the current dataset many key variables are very highly correlated with each other. For example, all TNT cards in the analysis sat in systems with exactly 128Mb of Ram. This means, by accident of sampling, that we could not get to see the effects of TNT across differing levels of system ram. In the case of the present ability of the model to “forecast” the existing data, this does not provide a major problem. But it might be a major handicap to forecasting accurately data outside the existing dataset, particularly for systems with considerably more or less than 128Mb of Ram.
Finally, the sample is simply not random. We don’t know how accurately it represents the “universe” of all identical hardware systems (let alone all non-identical ones). Errors we see in the current estimates, and they are there, cannot be assumed to be freely projected in exact replica to the world of systems “out there.”
Summary of Findings
Not surprisingly, the major factors influencing Falcon4’s fps continue to be: cpu speed (.06 fps per Mhz + an additional exponential quantity) gouraud (on/off)/sliders(right/left) (-10.9 fps if on/right) quantity of video ram (0.8 fps per Mb) quantity of system ram (.03 fps per Mb)
There are a number of notable hardware interactions, software interactions, and hardware-software interactions:
The impact of video ram decreases as the quantity of system ram increases eg: at 16 Mb video ram with 128Mb of system ram the hit is 4.30fps the impact of the TNT and Banshee cards clearly improves as cpu speed increases, but this is far less clear for the Voodoo cards eg: as cpu goes from 300Mhz to 450Mz, the TNT adds 3.8 fps, the Banshee 3.2 -the negative effect of gouraud on fps is greater at 640 x 480 pixels than at 800 x 600 e.g: for, say, a 300Mhz system, gouraud/sliders extracts a penalty of -10.9 at 640x480, but at 800x600, the penalty drops to –5.1 fps -faster cpus actually increase gouraud and pixel penalties on fps -compared to the other cards, the Voodoo2 in particular shines with gouraud on (+1.7fps), however, when gouraud is on, the TNT extracts a significant penalty on fps (-2.4fps)
NOTES
There is clear evidence that the positive effect of cpu on fps increases as cpu increases. This exponential effect was only briefly explored in the analysis.
For a “typical” system with 128Mb of Ram and 16Mb of video Ram, at 300Mhz, Voodoo2 is the clear fps winner over Banshee and TNT. But, at 450 Mhz, Banshee beats TNT by almost 1 fps and beats Voodoo2 by more than 2 fps!
The regression model estimated the data so well that the standardized departure of estimated fps from actual fps was less than 1 fps (0.963 to be exact). To see which cases seriously deviated from expected fps, go to Appendix 2 (the largest overestimate was 3.82 fps and the largest underestimate was 2.50fps).