User:Someonewithpc/GSoC2019/Log
Daily logs
I was only recommended starting a daily log somewhat late, so the early days are missing.
14/06
Today I finished refactoring code to move iterators to theirs own file, tested the code, profiled and benchmarked it. I started working on moving to not depending on Boost for iterators. While it does a remarkable job and arguably simplifies writing the code, it is harder to read and seems to cause some performance penalties, due to the somewhat high amount of indirections, which, since Blender is such a huge project and generally build with O2, the compiler seems to be unable to fully optimize. The use of straight loops needs to be evaluated, to see how much, if any performance is lost.
15/06
Today I explored how boost::memory_mapped_file is implemented and used and did some benchmarking to figure out how to best implement it.
17/06
Today I benchmarked the performance of multiple IO options and ported the OBJ exporter to use fprintf. I started working on making IO be done in blocks, which would make parallelization easier.
18/06
Today I started removing the use of boost::iterator_facade from the iterator implementation, which should both remove a dependency and make the core more readable and expandable. It will also, potentially, increase performance.
19/06
Today I finished removing the use of boost::iterator_facade and ran some benchmarks and got these results:
| Scene | 661c98ca146 (base) | b9e10ba1b62 (fprintf) | d08fbfacb21 (no boost::iterator_facade)
|
d08fbfacb21 (with duuplicates) |
|---|---|---|---|---|
| Default scene | 201ms | 31.4ms | 1ms | 1ms |
| Took | 46.1s | 43.1s | 40.9s | 11.6s |
21/06
Today I started working on exporting materials. I had initially started doing it in C++, but it was agreed that it would be better to use Python instead. I had some difficulties with building, for some reason, but deleting the build folder fixed it. It still took a while, though. I also re ran the previous benchmark in release mode (turns out I was using a debug build), and the current timing is 8.5 seconds.
23/06
Today I implemented material exporting in Python.
24/06
Today I implemented NURBS exporting, in C++. Aditionally I refactored most of the code so that the ExportSettings include only what's needed for most exporters, as well as defining only the needed RNA properties, so it doesn't clutter the Python interface.
25/06
Yesterday I was suddenly told I would have to leave my current apartment in three days, so I had to spend the day looking for an apartment, but I wasn't able to find one so far. A guy tried to scam me, even, so not great. I wasn't able to get any work done because I was really tired from running around all day.
26/06
Today I continued the apartment hunt and found a place :) More related to Blender, I created an updated schedule which contains the tasks I have done already as well as what I plan to do, updated as I go to match what's needed at the time. I spent quite some time trying to understand the problem with the normal_iter but I wasn't able to understand it yet. Decided to start by looking through Sybren's USD code, as per his suggestion, to understand how to properly use the depsgraph to iterate over objects and started implementing it to test wether it makes a difference to the current problem. Had to go to sleep before it worked.
27/06
Today I tried to solve a variety of bugs and tried to incorporate Sybren's suggestion of using DEG_OBJECT_ITER_BEGIN to iterate over objects as well as some small refactorings. However, I got stuck for a couple of hours getting weird results, because it seems the objects we get from this iterator don't retain the selected flag. Therefore I'm not entirely sure if this is the most suitable way to achieve this task, but I'm not sure what is. Also took a while to figure out a compiling issue where the solution turned out to be that DEG_depsgraph_query depends on BLI_iterator but doesn't include it. I added that include to the file. I tried to understand how to apply transforms to objects after getting them from the above iterator, where they're already evaluated, but I wasn't able to find out how yet. Asked my mentors but they were off for the day by this time.
28/06
Today I figured out that the selected flag is in fact kept, but it's stored in base_flag. OBJ export now applies scale and the object's transform, by individually multiplying the matrix. I feel like there must be a better way to do this, probably by somehow getting the depsgraph to handle all of this. Fixed crash when object doesn't have an UV map. Fixed two off by one bugs, since OBJ indices start at one. Fixed small memory leak.
01/07
Today I tried to understand the problems related to the normal_iter and to axis conversion. I wasn't able to understand why neither was working, but I fixed some small bugs and a memory leak. I tried googling a lot of stuff, but I wasn't able to figure it out and had to give up for the day.
02/07
Today I fixed a crash related to normal_iter. I'm not entirely sure how yet, but sometimes it would try to use iterators with a ridiculous size and then crash. I fixed the problem by mostly rewriting this iterator to directly use the underlying pointers, modeling it off of the code found in alembic which handles normals. I tried asking for help in blender.chat twice but my questions were essentially ignored. Still, I managed to figure this, but I expect there to be a better way, involving the depsgraph. I wasn't able to understand the problem with axis remapping.
03/07
Today I understood why there's an issue when importing obj resulting from exporting multiple objects: the sections are interweave in my exporter. I plan to fix this by pushing the iterator onto a vector for each object and then processing each section in sequence, at the end.
04/07
Today I tried to understand how to make axis remapping work as expected. I tried multiple things and couldn't get it to work, which was rather frustrating. I asked on #blender-coders, but it got ignored... I asked my mentors and they gave some feedback, but it still wasn't clear what I needed to do.
05/07
Today I finally understood how axis remapping is supposed to work. Blender uses Y as the forward axis, Z as the up axis and X as the left axis. Remapping simply consists of setting the forward and up axis as desired and then setting the X axis as the cross product of the other two. This produces a matrix which can then be multiplied with the object's transform.
06/07
Today I made it so that the OBJ exporter writes each type of data it writes grouped together; that is, all vertices, then all uvs, then all normals. I made it so it only writes the mtllib if it's needed. I fixed an indexing bug, which caused the importer to crash. I fixed a bug where an address of a temporary pointer was taken, which is Undefined Behaviour and would sometimes cause in a crash.
07/07 - 10/07
Due to some personal problems I was unable to work in this period. I discussed this with my mentors.
10/07
Today I started implementing smooth groups, but I'm getting a different result from what I get with the python exporter, not sure why yet. Found a bug and reported it on the chat and mont29 fixed it with 2e91fc3.
11/07
Today I started by doing some refactoring. I then moved on to trying to fix a bug related to custom normals, where it seems the wrong index is used and it leads to weird looking meshes. I finished support for smooth groups.
12/07
Today I added proper material support, where each mesh is tagged with the appropriate material. I did some more work on fixing normal exporting. There's still a problem with custom normals, it seems. I'm not sure what's causing it...
14/07
Today I kept looking for the issue with normal exporting, but I wasn't able to find it yet. JacquesLucke lent some help in trying to find the problem, but we're not sure what it is yet.
15/07
Today I added the UI for the OBJ Import and started analyzing how to best implement it. I'm using Alembic, LazyDodo's patch and HowardT's as reference.
16/07
Today I continued yesterday's task. Specifically, I added the boilerplate needed to add another operator.
18/07
Today I started trying to understand how to use Boost Spirit to parse OBJ files. I got stuck in what turned out to be a small detail which was hard to find in the manual: one can't use auto and must specify the actual type, because this is how it converts, for instance, an optional float to an actual float. This took me quite a while to figure out, though, which was kind of frustrating...
19/07
Today I started trying to understand how to create a new mesh with the correctly parsed data. I used the patches above as reference, but I must have done something wrong, because I was getting wrong results. I tried to figure out what was wrong, but I couldn't figure it out for now. I then noticed there was a segfault on parsing slightly more complex files. I then started investigating but didn't get anywhere yet.
20/07
Today I continued investigating the crash from yesterday. I eventually figured it out, and it seems to be the use of auto, even for rules which don't produce any result. It seems to be caused by a temporary object being created internally, and then the address of this temporary is taken, which is undefined behavior.
21/07
Today I continued working on the OBJ importer. I'm still not sure why this is, but after fixing the crash yesterday, nothing is parsing correctly, except for comments. I continued to investigate, but I got kind of stuck for a while. I made a small repl application for faster iterating for looking for the problem. Unfortunately the use of Spirit has affected compilation times, somewhat annoyingly, but I still think it's worth it for the code clarity and performance, over using scanf. However, maybe it would be worth it to use flex/yacc, or some other lexer generator.