There have been a bunch of questions raised in this thread and I'd like to answer as many as I can. I picked questions that came up multiple times, and questions or comments that were particularly on point. So here we go, somewhat in order of most asked:
1) Given what I know now, that this NTM effort was going to consume 10+ years of my life, do I wish I had pursued an empirical approach instead?
No. Absolutely not. I actually did pursue an empirical approach for 20 years, while at Papyrus and during the early years of iRacing. That taught me something important: an empirical approach gets slower and slower the more tires you try to model and the more cars you try to model. There are just too many variables, and to adjust them requires a lot of testing and feedback between changes. Inevitably someone will bring up an issue where the empirical model deviates from real tires, and that involves adding some new bit to the model, with more parameters, which now need to be added to all the tires and cars... more testing, feedback, ad nauseum.
With the physically based model, there has been a lot of work required to get to where we are now, but the development is getting faster, not slower. That's because the more we know, the easier it gets. There have been several times that the model produced a behavior that was surprising, but upon looking more closely at real data we realized that behavior was correct. Very often the numbers and concepts that we already know allow us to think more clearly about what we don't know. It's absolutely the right approach--I am stoked about the V7 model. Lots of things are right about it. Will it have some issues? Probably. It'll take a while to transition over to it as well, but it's a big step in the right direction.
As an example apart from tires, think about Ptolemy's epicycles as a way to describe the movement of the planets, versus Kepler's elliptical orbits around the sun. Ptolemy's model was definitely empirical. No way to figure out how many circles upon circles any particular planet needed, or how large those circles should be, or anything. It's just a massive curve fit against a model that has no basis in a theory of why. Kepler came up with a much better, simpler curve fit, though he also didn't know why. Sadly, he died thinking his 25+ years of work had been a complete failure because he hadn't been able to explain the divine order of the heavens. Isaac Newton, though, was aware of Kepler's work (amazingly, if you've ever read Kepler's work, which is... way out there) and it was one of the things that helped lead to his theory of gravitation. Look how much more progress resulted from Kepler and Newton's theory than from Ptolemy's. A good predictive theory beats big data any day of the week.
The Magic Formula is like Ptolemy's epicycles. It fits data well (sort of) and can predict with fair accuracy what needs to be predicted in most common situations. Our physically-based model is like Newton's theory, in that it is able to explain what's going on and even predict things we didn't know. Well, in fact it's way more sophisticated and interesting and way cooler than Newton's model of gravitation, but we've been able to stand on the shoulders of a lot more giants than Newton could, so he still wins the smartest guy award by, like, a long way. For example, Newton did all his important work before Euler was even born!! Plus we use Newton's theory of gravitation in the sim, and it works really well, so we have to give him that. Also, because calculus.
2) How do you go about validating all this work?
This is a great question, and unfortunately it has a much more complicated and unsatisfying answer than all of us would like. Ideal answer is: "For every element of this model we've devised experiments which have verified our theory, and we have published many papers in peer-reviewed journals, and our results have been independently corroborated in other laboratories around the world." Sorry, not that way. It's more like a mundane, everyday part of the process.
Fortunately, a great many scientists around the world have published books and papers in peer-reviewed journals which provide results that we can use to build our model. The most important thing to remember is that once we see a paper with some data, and we use that data to come up with a bit of theory, we can NOT use that same data to validate the theory. We have to find new data, which we haven't seen before. We often get data from auto manufacturers, or race teams, or even (rarely) tire companies, in addition to finding more books and more papers. Sometimes, we use data to theorize, sometimes we use it to validate. But not both. Data is often very dependent on the exact experiment conducted. Good papers will explain in great detail the conditions of the experiment reported and the materials used. Using that detail, I can reproduce that experiment virtually, sometimes with code, sometimes in a spreadsheet, and see what my model outputs. I have read a lot of papers reporting on dynamic experiments on rubber. Most of them provide the rubber recipe; if not, the data is of more limited value.
In creating any complicated simulation, there will always be a lot of unknown numbers. But the more we learn, the more those numbers become constrained to have a range of reasonable values. It's a bit like putting together a jigsaw puzzle. At first, it seems impossible, but once you begin to fit a few bits together, and the outline becomes clearer, the easier it gets. The process is always one of double, triple, quadruple checking against new data.
As a general principle, I'm always trying to eliminate any "magic" (i.e. inexplicable) numbers. Every number we use should have a basis in reality, either it was measured (and we know under what conditions), or it is derivable from a largely accepted physical theory. Some numbers are quite easy to derive, like the moment of inertia of the right front wheel about its spin axis. Some numbers much less so, like how hot will the right front carcass be after 20 laps at Charlotte? But many numbers are related to each other--the right front carcass temperature is related to the pressure build after 20 laps, and so if we know one of those numbers (we might learn pressure build from a race team), we can deduce at least a small reasonable range for the other. For a long time, the rolling drag for a tire was computed with an empirical model, because we had no way to really calculate what it was from first principles. And in order to make all the cars and tires end up with reasonable temperature and pressure builds we'd have to turn a magic knob, the Rolling Heat Multiplier, in order to get numbers that were in the right ballpark.
Now, we are able to compute rolling drag from the carcass model, and the material parameters in the carcass, which are also measurable. We'll retain the Rolling Heat Multiplier for now in case we miss the mark from time to time, but it had better turn out to be 1.0 (i.e. no change required to the model rolling drag) for almost every tire. If not, then that will be good information for us. Maybe it will be just about 1.2 for all the tires, in which case there is something that isn't being accounted for in the model, or maybe it will be some very different number for every tire, in which case I will start trying to figure out what I'm doing wrong. There aren't very many magic numbers like this left in the tire model. No magic numbers and all right answers = good model. In that sense, validation is a continuous part of the work.
Some validation certainly comes from "feel," and the feedback from real racers who should know how it feels. We do a lot of that sort of testing, too. But feel can be illusory, so we have to be careful. There is always lag in force feedback wheels, and a lack of seat-of-the-pants forces, which make controlling a slide more difficult than in real life. I always find that I'm over-driving the car in-sim and being too aggressive with the car (which I believe is fairly common among all of us). In the past, every time I'd do a race weekend, I'd be much faster back in the sim afterwards. Why? Because the real car would force me to be smoother and not over-drive it as much. Using that same approach in the sim is better. If it "feels right" in the sim, sometimes that means it's too forgiving of your (probable) over-driving. Once you get used to a more realistic, less forgiving feel, it's just as easy to control. You just have to be smoother and keep your eyes up so you can respond faster to over-rotation, or any other bad situation.
3) When will flat spots be modeled?
95% of the behavior of a lockup is modeled already. Nobody is being denied a championship because somebody else is locking up in every corner and paying no price for it. Modeling the remaining 5% is on the to-do list, but currently it comes after rain, which is another feature everybody wants--until they get it.
4) How much rubber curing is happening on track?
Not much at all, unless your carcass temps are greater than 265 Fahrenheit (130 C). This pretty much limits it to high speed oval racing, and even there it's just one effect out of several that reduce grip.
5) Tell us more about tire failure from excessive heat.
I don't want to over-hype this, it'll be some kind of rapid deflation event (i.e. inflation pressure goes to zero, just like current tire failures) based on exceeding some temperature for too long a time. The main thing is to provide a reasonable downside to running very low tire pressures. But even before failure, running pressures that are too low will in many cases not be fastest. Also, for now the behavior approaching, at, and beyond the critical speed can be a little weird due to some simplifications in the model. That is something I'm aware of and will try to address.
6) How reliable are lab results when reverse engineering rubber compounds?
These more provide a range of values, and constrain what's possible. We get a pretty good idea of how much of the compound is carbon black, less accurate is the determination of polymer vs. oil content. The lab tests give some raw data sweeps that are very useful, as well. Another double check is testing Shore A hardness of the tire's tread. But that depends on temperature, how long a reading (one second, three seconds, ten seconds?), and on the thickness of the tread, so care must be taken. Telemetry data is useful for grip determination (especially since our track geometry is close to exact), but aero forces need to be known precisely.
7) Why not update all the cars at once with the new tires?
Going to the V7 model is like having a new tire manufacturer come into a race series. Cars are often designed around the tires. Changing tire constructions and/or rubber compounds can change a car's handling quite a lot, even in the real world. So a lot of testing and setup work needs to be done. It's better to get feedback from a more limited number of cars at first. If there is something in the model that needs to be changed, we can do that without messing up setups for all the cars. Also, more work is necessary in the V7 code to properly do dirt. I think in the long run it will be better, but we'll probably stick with V6 on dirt for a while. Dirt masks the over-the-limit breakaway that is one of the biggest issues on pavement.
8 ) ...let's should be lets.
Oh jeez, I have to admit to being an apostrophe Nazi myself, so it's quite embarrassing to have let this slip through. Fixed! Thanks, Todd! Won't happen again...
9) How does this compare to the tire manufacturers' models, and would they be interested in what you're doing?
I don't really know the answer to this, but I can speculate. A large tire manufacturer has very different goals than a sim-racing developer. One, they don't need a model that runs in real-time. So they can use finite element analysis (FEM) extensively. For us, that's a little like ray-traced graphics from the movie industry. We may be able to get there some day, but probably there will always be a faster way to draw things. Tire manufacturers probably care the most about being able to sell more tires to a public that has little interest in driving at the limit. We are interested in selling more memberships to the limit-driving cognoscenti. The public buys tires based on the three things that are mandated to be measured by the government: grip rating (essentially how good the tire is at stopping while locked up at 40 mph on a wet road), mileage (miles per gallon of fuel burned, that is, essentially rolling drag on the highway), and wear rating (how many miles it will go before I have to buy a new one). Also, the public don't want their tires to pop when running over a beer can. On this, we can agree--racers don't want that, either. The public want tires that are not too noisy or harsh on the highway.
So much of the FEM work that tire companies do (but not all, I'm sure) has to do with: where and how does heat build up in the tire? What are the stresses at key points and how long can the tire materials last with that stress and heat? How can we better remove water from the tread area? How do we make the tread blocks quieter? If Joe Public doesn't pay any attention to his inflation pressures and is driving on the highway with 5 psi in a rear tire, how do we protect him from himself? And us from liability? How do we simultaneously increase wet grip, decrease rolling drag, and make the tire last longer? Can we do all this with cheaper materials? And of course, can we make a really cool video to run during the Super Bowl? (preferably with a little baby being saved by our FEM model graphics)
We care about the first (how and where of heat build up), but much less about the others. My guess is the tire companies are already able to get better numbers for their purposes than we are. But we are better at getting numbers for our purposes than they are.
10) Have you seen Nicki Thiim's video?
Yes. And I have to say, Nicki's videos are as much fun to watch as it is to listen to Jens Voigt announcing a bike race. In other words, awesome! And to be honest, he is right about the V5 slip and grip over the limit model. Although, I think it is not as bad as he makes it sound. Ok, it is if it's hot out. But he also says that he loves the "curb-surfing" (a great term) and that has much to do with the V6 carcass stiffnesses, which I think are very good. And he liked everything else about iRacing, including the graphics and track geometry realism. I think he's right about that, too.
11) What is the sampling rate of your physics model?
360 Hz, is the short answer. But we calculate forces twice per update step, so we do player tire force calculations at least 4x2x360 = 2880 times per second. More if some of the tires are contacting multiple surfaces (i.e. curbs). We use IEEE floats.