notes.txt

HOW THINGS WORK
-so deferred shading is all about reducing the number of fragment shadings that need to happen
-so the deferred shader rasterizes, and then performs lighting computations only for what's visible
-in webgl apparently this requires writing out buffers of what's visible
-so these are the "multiple render passes"
-btw, shader programs can have inputs and outputs bound on init

BLOOM
-something's... not so right
-but the idea of this should work
-just need a better filter system
-should we make this a separate post processing shader? can we do this without addtl buffers?
	-yes. probably. hopefully.
	-which would make profiling later easier.
-TODO:
	-add camera position uniform to blinn-phong [done. turns out this was in Kai's updates]
	-find a better filter [done]
	-make this a separate post-processing shader

SCISSOR TEST [done]
-ok ok ok. so it's restricting the shading to a screen space "box" that gets a lighting computation
-shouldn't be too bad
-we'll need to tweak ambient, btw. scene is too dark atm. [done]
-TODO:
	-enable and use Kai's primitive scissor test [done]
	-add scissor test debug mode [done]

PACKING AND OPTIMIZATION -> all about trying to do this stuff while keeping it toggleable
-try 2-component normals
	-use length of x and y to get magnitude of z
	-these are normalized, so no component will be greater than 1 -> use that to pack sign of z
	-how to do this while making it toggleable? -> probably needs a new shader

-apply normal map and pass it along -> easy, probably. but will interfere with debug -> needs a new shader

-reconstruct world space pos w/cam matrix, x/y/depth -> hmmm. -> needs a new shader
	-need to invert cam matrix? -> look up how to do this.
	-we probably already need to add a uniform for cam position, b/c blinn phong is broken without it

2 COMPONENT NORMALS and WORLD SPACE RECONSTRUCT
-so we don't want to change from buffers of vec4s
-which means if we're packing, we want to pack both 2-comp normals and world space pos into one buffer

-so we'll need a new copy.frag.glsl and a new copy.vert.glsl
	-new copy.vert.glsl will need to push screen space position over
	-new copy.frag.glsl will only do 2 buffers: texture2D and compacted position/normal
	-new copy.frag.glsl will also need to apply surface normal internally

-we'll need to add a new blinnphong-pointlight.frag.glsl and ambient.frag.glsl as well
	-just need to add position/normal reconstruction

-which means we'll need to add setup for all of these in deferredSetup.js
-might as well add a new debug.frag.glsl as well to help debug reconstruction
-so we're basically duplicating a huge chunk of the project. ok.

SCREEN SPACE MOTION BLUR and/or TOON SHADING
-screen space motion blur has a GPU gems article, so this might be more straightforward
	-would also be post processing
-but toon shading: -> TOON SHADING IS COOL
	-need to sample at each frag for depth. if depth change is beyond some threshold, then need to add a line
	-ramp shading: like cutting up the gradient of blinn-phong for specular. we can probably generate this.
	-so this has to be an alternate shader to blinn-phong


TILING
-basically trying to reduce number of light calculations by tiling
-think about how to do this
-seems like it could extend from scissor test?

TODO before seeing Kai:
-make bloom toggleable [done]
-make scissor test toggleable [done...? there was a mystery crash]
-add a toon shader. make it toggleable. should theoretically be very similar to blinnphong in the js

Seeing Kai:
-so I'm trying to have a separate pipeline that I can toggle
-this pipeline has copy stuff that produces fewer gbuffers
-my understanding is that each framebuffer can have its own gbuffers
-but the gbuffers here seem to be wrong. what's up?

TILING FOR REAL
-so the basic idea is to store each light's attributes in textures
-then you store each tile's lights list in another texture
-and to render a tile, you run the shader over that tile region with the index and length of the tile's light list

QUESTIONS
-how to build the tile/light lists?
	-likely involves buffering data?
	-ok. lists: can use texImage2D with internal format UNSIGNED_INT, format DEPTH_COMPONENT. presumably we can upload an unsigned int array.
		-https://www.khronos.org/registry/webgl/extensions/WEBGL_depth_texture/
	-light info: will probably have to be vec4 for pos/radius and vec4 color
		-will need to update per render with new position
	-we'll need a way to compute a "new UV" in the shader per light index
	-can PROBABLY use texture2D in the shader and pass in an appropriate UV value

-how to get the lighting shader to loop over lights?
	-define some max number of lights?
	-glsl for loops have to have a determined number of iters right?
-how to get the lighting shader to only handle the tile it's in?
	-scissor? -> yeah yeah yeah!
	-NO. we can render all the tiles at once.
	-this necessitates a light data per tile list texture

LOOK AT
-binding textures to uniform pos: util.js -> window.readyModelForDraw
	-it's just a matter of binding texture to uniform
	-be careful with the whole active texture thing?
-also: deferredRender.js -> bindTexturesForLightPass
-reloading textures: util.js -> handleTextureLoaded
	-there's a "texImage2D" use here.
	-MSDN says it can be used with int arrays -> ArrayBufferView
	-https://developer.mozilla.org/en-US/docs/Web/API/ArrayBufferView
	-http://www.javascripture.com/ArrayBufferView
	-so apparently ArrayBufferView just means "some array"
	-so straight up Uint16 arrays are ok?
	-http://stackoverflow.com/questions/9046643/webgl-create-texture

TRY
-overwriting the color pass texture in bindTexturesForLightPass with a float array
-making a texture the same way as in framework's loadmodels. can we inspect it?
	-no

TODO
-add a texture of light info (color, position + radius)
-add an int texture of light lists -> numLights * numTiles size. biiiiig.
-add an int texture of light list pointers (per tile thingy) -> actually, can we just bind these as uniforms? lol why not?
-add a light shader that does blinn phong over each light in the list
	-needs to compute UV from indices provided, so needs to know how many lights
	-needs to have enough looping room to do all the lights possible -> uh oh
		-introduces a restriction on how many lights we can do per pass
		-should we care about blending? or just artificially limit?
		-presumably we're hoping you won't have > 100 lights in one tile
	-aaaaand all the stupid messy binding and stuff that's associated with this

new understanding:
-ok. so the light data structure has to be bound to the fbo
-but how is this done?! it seems all textures attached to the fbo must be the same dimensions as the fbo!
-as long as max_lights_per_tile < tile_width, this can still be done... hmmm

Steps
1) set up a new copy pass for tiling [done, untested]
	-include a new gbuf for each element in the datastructure
2) write the new copy pass [done, untested]
	-must also overwrite the appropriate gbuffers with light datastructure
	-this will heavily change how indexing is handled
3) write a new bindTexturesForLightPass [done]
4) write the debug_tile render pass [done]
5) write a new debug shader [done]

notes:
-to simplify, we're going to clamp the number of lights to resolution width
	-simplifies indexing when reading a light's params
-to reduce lookups: since we get a huge amount of memory for a sparse amount of data, let's have each light list literally store the light data.
-will simplify lookup in shader

PROBLEMS
-so at the moment, the limit on number of tiles means that some lights very obviously end up missing from tiles
-let's add an option to toggle a different kind of light input: it will sort the lights based on distance along the camera's view and then bin them using that sort

-wtf is going on with github.io? how do we do stuff there? do we have to host the same code in 2 places on github?!

TODO:
-add appropriate global structures
-figger out how to z sort -> transformations? durrr hur hur hur
-look at js sort: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort

-so light sorting is only sometimes useful. glad I made it a toggleable

TODO: [done] -> but there's still some artifacting, possibly remnant of scissor test?
-improve indexing so that we can support more lights per tile. 0.5 * TILESIZE * TILESIZE - 1 is ideal
	-this shouldn't be that hard
	-we can devote half the tile to colors and half the tile to positions, for better visualization
	-will need to remove the fake global light list. who needs it anyway?

G-buffer format change performance:
-10 lights: 32 ms 30 ms
-20 lights: 65 ms 40 ms
-40 lights: 110 ms 82 ms
-80 lights: 210 ms 130 ms
-160 lights: 402 ms, 

scissor test:
-10 lights: 15 ms 16 ms
-20 lights: 20 ms 16 ms
-40 lights: 30 ms 18 ms
-80 lights: 49 ms 28 ms

tiling
-10 lights: 25 ms
-20 lights: 27 ms
-40 lights: 30 ms
-80 lights: 47 ms
-160 lights: 402 ms, 80 ms, 50 ms

80 lights:
16x16: 80 ms
32x32: 47 ms
64x64: 25 ms
128x128: 28 ms
256x256: 32 ms

bloom, stock pipeline:
20 lights: 65 ms, 67 ms
40 lights: 116 ms, 117 ms
80 lights: 210 ms, 212 ms