I can't do much about the shadows unfortunately, it seems there's different shadows just as with Oblivion so you have tree shadows, world shadows, item shadows and character shadows.
Unfortunately despite resolution adjustment, tweaks and other commands some of these will be a bit split apart and very jaggy, overly softened or what to describe them as.
Especially item shadows.
(Not all items appear to cast shadows though, this could also mean that more advanced behavior is controlled either via ESM code - mod potential - or unfortunately via the NIF file itself - as how it worked in earlier titles - and checking thousands of NIF files would be quite a task but maybe it's possible to improve the situation.)
Currently re-adjusting the configuration I've been toying with, since there's already LOD low-detail landscape it's not necessary to render certain things all the way to the horizon (I probably got even further with those values though.) so I've scaled back certain things again, it goes a bit back and forth but I get a better understanding of how things work as I toy around with it, much more stable also which is the same problem highly tweaked Oblivion or Fallout 3 / NV setups faced.
(The part about the game using 1.2 - 1.7 GB of VRAM already will also hit third party retextures hard but we'll see how that goes I guess.)
EDIT: Skyrim.ini
uExterior Cell Buffer=72 - I figured a doubled buffer would be OK, it might not actually do much but it doesn't seem to have any negative effects either.
fShadowLODMaxStartFade=5000.0
fSpecularLODMaxStartFade=5000.0
fLightLODMaxStartFade=5000.0
Increase these to the same value for more uniform cut of lines for shadows, specular highlights and light effects, above max but not too much of a increase, you won't really see those tiny distant objects much differently so I halved these from what I had earlier.
And that's it for Skyrim.ini which only holds a few settings.
SkyrimPrefs.ini
iBlurDeferredShadowMask=6
This one needs testing but I assume it adds more steps to the effect and thus smooths it out, doubled as to begin with.
(At worst it's a 1 - 3 value with low, medium and high with 0 for off.)
fInteriorShadowDistance=6000.0000
fShadowDistance=16000.0000
Double shadow draw distance, unsure what the unit is but as I said earlier I notice nearly no performance effect (CPU limited?) so I tend to go for better image quality where possible.
iShadowMapResolutionSecondary=2048
iShadowMapResolutionPrimary=4096
Higher seemed redundant as shadows don't really change as much as expected but it can certainly be set to double this if needed.
iShadowSplitCount=4
Similar to the above about shadows, was hoping this split them up better but maybe that's a issue instead of a improvement if so?
(Still it increases as image quality does as can be seen when comparing the preset ini files in the game folder.)
fDecalLOD2=5000.0000
fDecalLOD1=2500.0000
More decals further away, blood effects and what not.
(Not exactly doubled but I rounded it out to this.)
fSpecularLODStartFade=5000.0000
fShadowLODStartFade=1000.0000
fLightLODStartFade=5000.0000
Similar to the ones in Skyrim.ini but shadow LOD was shorter so I kept that.
(But still made a bit increase from the default 200)
iWaterMultiSamples=4
iMultiSample=0
Ultra/V.High ini preset has that at 4 instead of 0 and it should smooth out water edges a bit but the soft water effect isn't used everywhere (Mostly only on larger masses of water.) so it might not be that noticeable.
MSAA caused a transparency glitch on some objects so it got disabled, goes from 0 to 16 albeit capped at what the GPU card supports of course, usually 8
bTreesReceiveShadows=1
bDrawLandShadows=1
Trees can have shadows on them (Bit more demanding but pretty.) and I don't really know about that last one, was hoping for shadow effects near raised terrain or dug out terrain but it might be something else, still it was worth enabling.
fMeshLODLevel2FadeTreeDistance=10000000.0000
fMeshLODLevel1FadeTreeDistance=10000000.0000
fMeshLODLevel2FadeDist=10000000.0000
fMeshLODLevel1FadeDist=10000000.0000
These cap at this value (V.High preset ini is a bunch of 9's but it gets rounded of to this when you start the game, tree distance was set same as the rest instead of being significantly shorter, mostly only covers the LOD low-detail trees anyhow and those don't affect framerate at all here.)
iMaxSkinDecalsPerFrame=50
iMaxDecalsPerFrame=250
More splatter when slashing stuff basically but also goes for other effects.
sD3DDevice="AMD Radeon HD 6900 Series"
Your GPU model here, resets to auto-detect if incorrect so copy over what your file uses to avoid that.
bFXAAEnabled=1
iShadowMapResolution=4096
I use FXAA as to get a form of AA at all but it adds some subtle blur instead, unsure if the .dll hack method is better but this works fine.
(Also a bump in shadow resolution can be done but I think 8192 is a tad too high so I kept it at default, can go as high as the card supports for these of course but don't expect much of a difference.)
fShadowBiasScale=0.1000
This bugged shadows on very low values but decreased as options were set higher and higher so I tried a bit lower than the default of 0.150 for some improvements to shadow detail.
iShadowMaskQuarter=8
Again one of those more unknown shadow settings but as it increased on V.high and that I added a bit extra here too.
iPresentInterval=0
No vsync also improves menu responsiveness and some say general mouse movement, especially in first person specifically but I just use it as I prefer it disabled and don't really notice screen tearing much at all.
bTransparencyMultisampling=1
I don't use MSAA but I kept this enabled anyway, might improve visuals without using FXAA when MSAA is enabled as to cover vegetation and similar.
iActorShadowCountInt=8
iActorShadowCountExt=8
These were in the preset files but not added, added it back (Bottom of [display] ) and doubled them, shadows from all eight basic directions sort-of, or so I had in mind.
b30GrassVS=1
If it's already using 3.0 pixel shader why not 3.0 vertex shader as well, no reason to keep it disabled I imagine.
fGrassStartFadeDistance=15000.0000
fGrassMaxStartFadeDistance=15000.0000
A little more than doubled but sets grass to render further out, looks better and the grass is pretty low-detail so it doesn't do much for performance either.
(Grass really hit Oblivion in terms of performance though.)
bGamepadEnable=0
Don't use one so this system got disabled.
bCrosshairEnabled=1
fHUDOpacity=1.0000
Untouched and you can set crosshair in the game options but here you can also set menu transparency, I guess 0.0000 is fully solid.
bSaveOnPause=0
bSaveOnTravel=0
bSaveOnWait=0
bSaveOnRest=0
I save manually using the console command "Save Savename" and "Load Savename" so I disabled all of this.
bShowFloatingQuestMarkers=1
bShowQuestMarkers=1
iDifficulty=4
Mostly untouched but I play on master level difficulty, might want to lower it.
bDialogueSubtitles=1
bGeneralSubtitles=1
bShowCompass=1
I like the subtitles for clarity and you can disable the compass here if you would want to but finding your way to the quest location might be very difficult without clear descriptions.
bMouseAcceleration=0
This shouldn't even exist so away with mouse acceleration, more precise now.
iMaxDesired=2500
More particle effects (Spells and such?) please so I increased this.
fTreeLoadDistance=100000.0000
fBlockMaximumDistance=500000.0000
fBlockLevel1Distance=100000.0000
fBlockLevel0Distance=50000.0000
fSplitDistanceMult=3.0000
The defaults I used earlier caused stability problems so I kind of doubled existing values here, fully stable so far and better draw distance for objects.
uiMaxSkinnedTreesToRender=50
I don't quite know what it does since it's name UI unless "u" is a typo, more fully detailed trees I guess, default was 20 so I boosted it up.
uMaxDecals=1000
More decals if possible, can hit performance a bit.
uMaxSkinDecals=250
uMaxSkinDecalsPerActor=100
As above but for characters.
fLODFadeOutMultObjects=50.0000
fLODFadeOutMultItems=50.0000
fLODFadeOutMultActors=50.0000
Objects, items and character draw distance got a bit of a improvement from this, pretty hefty but very effective.
iWaterReflectHeight=2048
iWaterReflectWidth=2048
Better water reflections, not that they reflect much of anything due to their appearance but still.