2wKdvQpUKruG0PeUl2PKIV changeset

Changeset333464343434 (b)
ParentNone (a)
ab
0+
0+[11:04:01] <ploum> Hello
0+[11:04:18] <ploum> I've written my own treemodel to connect to my own data model
0+[11:04:52] <ploum> But my data model run in another thread than the gtk.mainloop
0+[11:04:59] *** foser has joined the room as a participant and a member
0+[11:05:49] <ploum> to not have any problem with threads, I decided that my model will send modification signals to the treemodel using gobject.idle_add
0+[11:05:59] <Company> have fun marshalling :)
0+[11:06:26] <ploum> problem is that, sometimes, the treemodel explore the tree by itself thus, when receiving the signals, duplicating some actions
0+[11:06:34] <ploum> (rows are added or removed twice)
0+[11:07:03] <ploum> what's the good way of implementing such a system ?
0+[11:08:41] <kamstrup> ploum, I can't think of anything other than keeping track of which signals you already have queued idle handlers for and then making sure you only ever queue one per change
0+[11:08:55] <kamstrup> it wont be particularly easy to get right though :-)
0+[11:10:42] *** gnumdk has joined the room as a participant and a member
0+[11:10:47] <gnumdk> Hello
0+[11:10:59] <ploum> kamstrup: could you please explain a bit ?
0+[11:11:01] <gnumdk> Is there a way to get real rounded menu in gtk ?
0+[11:11:41] <ploum> I've tried something similar but it didn't work. Hopefully your idea is different
0+[11:12:19] <kamstrup> ploum, ok... so it's basically some hashmap of operation-id -> idle_handler_id
0+[11:12:27] <kamstrup> g_idle_add returns a source id
0+[11:12:44] <kamstrup> so you need to construct some id for a given change as well
0+[11:13:25] <kamstrup> like if you change a row you could have something like "CHANGED"+row_id
0+[11:13:38] <kamstrup> then put that in a map
0+[11:13:41] <ploum> I think it is what I did. The problem is that *sometimes*, treeview discover by itself, sometimes not, but there's no way to know what treeview has discovered.
0+[11:13:54] <Company> uhm
0+[11:14:04] <kamstrup> hmmm - ok, there might very well be some details I haven't grokked
0+[11:14:26] <Company> by design, you cannot have a half-updated treeview
0+[11:14:41] <Company> your treeview must always be correct wrt the signals you emitted
0+[11:14:56] *** bpeel_away is now known as bpeel
0+[11:14:57] <Company> so adding two rows, then emitting two "row-added" doesn't work
0+[11:15:00] *** Milosz has left the room (Ping timeout: 600 seconds (~AKKG@91-64-178-209-dynip.superkabel.de))
0+[11:15:15] <Company> you need to add a row, emit row-added, add next row, emit row-added
0+[11:15:27] <tristan> ploum, you say that "model will send modification signals to the treemodel using gobject.idle_add"... where is the difference between the model and treemodel ?
0+[11:16:13] <tristan> eh, /me still unsure how you can do it safely without caching the actual changes somewhere
0+[11:16:36] <ploum> tristan: the model is not gtk related at all. It's something that runs in its own thread and offer a tree API. My treemodel only connect to it.
0+[11:17:26] <tristan> ploum, your GtkTreeModelIface implementation offers a model viewable by a treeview... and accesses your unrelated datamodel as a thin layer correct ?
0+[11:17:42] <ploum> tristan: indeed
0+[11:18:34] <tristan> ploum, so when the treeview invokes methods of the GtkTreeModel implementation directly, that implementation needs to report out-of-sync data sometimes
0+[11:18:56] <tristan> even if it knows the data has changed under the hood, but is pending an idle handler to find out
0+[11:18:59] <ploum> according to Company advice, maybe I could have a huge cache, take the values from there but reset this cache each time I receive a signal ?
0+[11:19:28] <tristan> ploum, if you cache the whole data, it defeats the purpose of a custom model though correct ?
0+[11:19:30] <tristan> heh
0+[11:19:40] <ploum> tristan: yes, indeed
0+[11:20:13] <ploum> but I don't see how to report out-of-sync data otherwise
0+[11:20:47] <tristan> so, when publishing a change from the thread, you need to lock the treemodel / save the old treemodel state for that row in some queue / unlock the treemodel
0+[11:21:15] <tristan> then when emitting signals... you need to unqueue those changes one by one while taking care of the lock
0+[11:21:23] *** hansfbaier has left the room (hansfbaier!~jack@61.5.86.211 PART #gtk+ (~jack@61.5.86.211))
0+[11:21:29] <tristan> and when the GtkTreeModel is invoked by the TreeView directly
0+[11:21:56] *** andre_ has joined the room as a participant and a member
0+[11:22:00] <tristan> it has to be sure to use the queued "inconsistent data" instead where available (i.e. the idle hasnt passed through yet)
0+[11:22:14] <tristan> could be something accumulative like that
0+[11:22:22] *** lx has joined the room as a participant and a member
0+[11:22:54] <ploum> I never thought it would be so painful :-(
0+[11:22:59] <tristan> heh
0+[11:23:36] <tristan> ploum, you even have to consider that the same data may change multiple times before the treeview has time to get the change for that data
0+[11:24:13] <tristan> ploum, however just the thought of a cross-threaded model implementation makes my brain hurt ;-)
0+[11:25:01] <tristan> ploum, it can be much simplified if the nature of your data is specific, for instance a threaded log view is easy... it only has to append rows asynchronously
0+[11:25:59] *** mcrha|afk is now known as mcrha
0+[11:27:17] <ploum> tristant: I fail to see how it is easy ?
0+[11:27:24] <ploum> the problem is the same, isn't it ?
0+[11:30:15] <tristan> ploum, not really - if you are just accumulating incoming server logs or such, then the only cross-threaded operation is "add-row"
0+[11:30:43] <tristan> in that case you queue the data in an idle handler and append it to a model that's safely accessible from the gui thread
0+[11:31:10] <ploum> here, I've a complex tree and I use add, remove and update
0+[11:31:17] <tristan> ploum, i.e. you are not taking into accound row-changes interleaved with inserted rows, mangled row indexes and all
0+[11:32:06] <ploum> well, in my model, I pay a lot of attention to that
0+[11:32:07] <tristan> ploum, well then your case is more complex than an ever growing server log that needs to be viewed ;-)
0+[11:32:08] <tristan> heh
0+[11:32:44] <ploum> thanks a lot for all the information
0+[11:32:56] <ploum> It makes me a bit depressive but, at least, it's informative
0+[11:33:21] <ploum> cannot we imagine making the treeview a bit more intelligent ?
0+[11:33:43] <tristan> ploum, I've been imagining that recently
0+[11:33:58] <Company> ploum: theading in general is hard
0+[11:34:05] <tristan> ploum, but my imaginings havent included accessing models in other threads
0+[11:34:21] <Company> ploum: if you have to synchronize state between 2 threads, you're always in a world of pain
0+[11:34:22] *** kamstrup has left the room (Ex-Chat (~kamstrup@188.114.151.76))
0+[11:34:46] <Company> it's what intel figured out when they had to sync CPU caches of multiple CPUs to the same data ;)
0+[11:35:25] <ploum> Company: indeed, but you cannot decently imagine doing all the work in the same thread as the GUI !
0+[11:35:41] *** Milosz has joined the room as a participant and a member
0+[11:35:52] <Company> ploum: no, but you need an intelligent way to store the data that is accessed by the GUI
0+[11:36:13] <Company> i.e. in MVC, decide where you keep the M if V and C are in different threads
0+[11:36:14] <tristan> ploum, right, we usually encourage people to at least keep all the GTK+ related calls in the same thread though
0+[11:36:15] *** kamstrup has joined the room as a participant and a member
0+[11:37:31] *** xan has joined the room as a participant and a member
0+[11:37:48] <ploum> I choosed to put the M and C in the same thread. But it doesn't work…
0+[11:38:27] <ploum> tristan: all the gtk calls are in the same thread, that's why I'm using idle_add. If you don't do that, you have very very naughty crashes ;-)
0+[11:39:21] <tristan> ploum, are they ? what if the GtkTreeView accesses your model implementation and requires it to consult data that may have been asynchronously updated ?
0+[11:39:30] <tristan> its hard to say
0+[11:40:12] <tristan> ploum, on the other hand, you might try "the other approach", which is typically more dangerous
0+[11:40:19] <tristan> but in your case possibly suitable
0+[11:40:29] <tristan> ploum, scratch the g_idle_add calls alltogether
0+[11:40:42] <tristan> and aquire the gdk_threads_mutex() when updating the model internally ?
0+[11:41:00] *** andreasn has joined the room as a participant and a member
0+[11:41:15] <ploum> tristan: that's an option, indeed
0+[11:41:19] <ploum> it worths trying
0+[11:41:20] <tristan> ploum, this may have problems that I'm not considering at the moment
0+[11:41:27] <tristan> I would try it yeah
0+[11:41:32] <ploum> thanks a lot
...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
--- Revision None
+++ Revision 333464343434
@@ -0,0 +1,96 @@
+
+[11:04:01] <ploum> Hello
+[11:04:18] <ploum> I've written my own treemodel to connect to my own data model
+[11:04:52] <ploum> But my data model run in another thread than the gtk.mainloop
+[11:04:59] *** foser has joined the room as a participant and a member
+[11:05:49] <ploum> to not have any problem with threads, I decided that my model will send modification signals to the treemodel using gobject.idle_add
+[11:05:59] <Company> have fun marshalling :)
+[11:06:26] <ploum> problem is that, sometimes, the treemodel explore the tree by itself thus, when receiving the signals, duplicating some actions
+[11:06:34] <ploum> (rows are added or removed twice)
+[11:07:03] <ploum> what's the good way of implementing such a system ?
+[11:08:41] <kamstrup> ploum, I can't think of anything other than keeping track of which signals you already have queued idle handlers for and then making sure you only ever queue one per change
+[11:08:55] <kamstrup> it wont be particularly easy to get right though :-)
+[11:10:42] *** gnumdk has joined the room as a participant and a member
+[11:10:47] <gnumdk> Hello
+[11:10:59] <ploum> kamstrup: could you please explain a bit ?
+[11:11:01] <gnumdk> Is there a way to get real rounded menu in gtk ?
+[11:11:41] <ploum> I've tried something similar but it didn't work. Hopefully your idea is different
+[11:12:19] <kamstrup> ploum, ok... so it's basically some hashmap of operation-id -> idle_handler_id
+[11:12:27] <kamstrup> g_idle_add returns a source id
+[11:12:44] <kamstrup> so you need to construct some id for a given change as well
+[11:13:25] <kamstrup> like if you change a row you could have something like "CHANGED"+row_id
+[11:13:38] <kamstrup> then put that in a map
+[11:13:41] <ploum> I think it is what I did. The problem is that *sometimes*, treeview discover by itself, sometimes not, but there's no way to know what treeview has discovered.
+[11:13:54] <Company> uhm
+[11:14:04] <kamstrup> hmmm - ok, there might very well be some details I haven't grokked
+[11:14:26] <Company> by design, you cannot have a half-updated treeview
+[11:14:41] <Company> your treeview must always be correct wrt the signals you emitted
+[11:14:56] *** bpeel_away is now known as bpeel
+[11:14:57] <Company> so adding two rows, then emitting two "row-added" doesn't work
+[11:15:00] *** Milosz has left the room (Ping timeout: 600 seconds (~AKKG@91-64-178-209-dynip.superkabel.de))
+[11:15:15] <Company> you need to add a row, emit row-added, add next row, emit row-added
+[11:15:27] <tristan> ploum, you say that "model will send modification signals to the treemodel using gobject.idle_add"... where is the difference between the model and treemodel ?
+[11:16:13] <tristan> eh, /me still unsure how you can do it safely without caching the actual changes somewhere
+[11:16:36] <ploum> tristan: the model is not gtk related at all. It's something that runs in its own thread and offer a tree API. My treemodel only connect to it.
+[11:17:26] <tristan> ploum, your GtkTreeModelIface implementation offers a model viewable by a treeview... and accesses your unrelated datamodel as a thin layer correct ?
+[11:17:42] <ploum> tristan: indeed
+[11:18:34] <tristan> ploum, so when the treeview invokes methods of the GtkTreeModel implementation directly, that implementation needs to report out-of-sync data sometimes
+[11:18:56] <tristan> even if it knows the data has changed under the hood, but is pending an idle handler to find out
+[11:18:59] <ploum> according to Company advice, maybe I could have a huge cache, take the values from there but reset this cache each time I receive a signal ?
+[11:19:28] <tristan> ploum, if you cache the whole data, it defeats the purpose of a custom model though correct ?
+[11:19:30] <tristan> heh
+[11:19:40] <ploum> tristan: yes, indeed
+[11:20:13] <ploum> but I don't see how to report out-of-sync data otherwise
+[11:20:47] <tristan> so, when publishing a change from the thread, you need to lock the treemodel / save the old treemodel state for that row in some queue / unlock the treemodel
+[11:21:15] <tristan> then when emitting signals... you need to unqueue those changes one by one while taking care of the lock
+[11:21:23] *** hansfbaier has left the room (hansfbaier!~jack@61.5.86.211 PART #gtk+ (~jack@61.5.86.211))
+[11:21:29] <tristan> and when the GtkTreeModel is invoked by the TreeView directly
+[11:21:56] *** andre_ has joined the room as a participant and a member
+[11:22:00] <tristan> it has to be sure to use the queued "inconsistent data" instead where available (i.e. the idle hasnt passed through yet)
+[11:22:14] <tristan> could be something accumulative like that
+[11:22:22] *** lx has joined the room as a participant and a member
+[11:22:54] <ploum> I never thought it would be so painful :-(
+[11:22:59] <tristan> heh
+[11:23:36] <tristan> ploum, you even have to consider that the same data may change multiple times before the treeview has time to get the change for that data
+[11:24:13] <tristan> ploum, however just the thought of a cross-threaded model implementation makes my brain hurt ;-)
+[11:25:01] <tristan> ploum, it can be much simplified if the nature of your data is specific, for instance a threaded log view is easy... it only has to append rows asynchronously
+[11:25:59] *** mcrha|afk is now known as mcrha
+[11:27:17] <ploum> tristant: I fail to see how it is easy ?
+[11:27:24] <ploum> the problem is the same, isn't it ?
+[11:30:15] <tristan> ploum, not really - if you are just accumulating incoming server logs or such, then the only cross-threaded operation is "add-row"
+[11:30:43] <tristan> in that case you queue the data in an idle handler and append it to a model that's safely accessible from the gui thread
+[11:31:10] <ploum> here, I've a complex tree and I use add, remove and update
+[11:31:17] <tristan> ploum, i.e. you are not taking into accound row-changes interleaved with inserted rows, mangled row indexes and all
+[11:32:06] <ploum> well, in my model, I pay a lot of attention to that
+[11:32:07] <tristan> ploum, well then your case is more complex than an ever growing server log that needs to be viewed ;-)
+[11:32:08] <tristan> heh
+[11:32:44] <ploum> thanks a lot for all the information
+[11:32:56] <ploum> It makes me a bit depressive but, at least, it's informative
+[11:33:21] <ploum> cannot we imagine making the treeview a bit more intelligent ?
+[11:33:43] <tristan> ploum, I've been imagining that recently
+[11:33:58] <Company> ploum: theading in general is hard
+[11:34:05] <tristan> ploum, but my imaginings havent included accessing models in other threads
+[11:34:21] <Company> ploum: if you have to synchronize state between 2 threads, you're always in a world of pain
+[11:34:22] *** kamstrup has left the room (Ex-Chat (~kamstrup@188.114.151.76))
+[11:34:46] <Company> it's what intel figured out when they had to sync CPU caches of multiple CPUs to the same data ;)
+[11:35:25] <ploum> Company: indeed, but you cannot decently imagine doing all the work in the same thread as the GUI !
+[11:35:41] *** Milosz has joined the room as a participant and a member
+[11:35:52] <Company> ploum: no, but you need an intelligent way to store the data that is accessed by the GUI
+[11:36:13] <Company> i.e. in MVC, decide where you keep the M if V and C are in different threads
+[11:36:14] <tristan> ploum, right, we usually encourage people to at least keep all the GTK+ related calls in the same thread though
+[11:36:15] *** kamstrup has joined the room as a participant and a member
+[11:37:31] *** xan has joined the room as a participant and a member
+[11:37:48] <ploum> I choosed to put the M and C in the same thread. But it doesn't work…
+[11:38:27] <ploum> tristan: all the gtk calls are in the same thread, that's why I'm using idle_add. If you don't do that, you have very very naughty crashes ;-)
+[11:39:21] <tristan> ploum, are they ? what if the GtkTreeView accesses your model implementation and requires it to consult data that may have been asynchronously updated ?
+[11:39:30] <tristan> its hard to say
+[11:40:12] <tristan> ploum, on the other hand, you might try "the other approach", which is typically more dangerous
+[11:40:19] <tristan> but in your case possibly suitable
+[11:40:29] <tristan> ploum, scratch the g_idle_add calls alltogether
+[11:40:42] <tristan> and aquire the gdk_threads_mutex() when updating the model internally ?
+[11:41:00] *** andreasn has joined the room as a participant and a member
+[11:41:15] <ploum> tristan: that's an option, indeed
+[11:41:19] <ploum> it worths trying
+[11:41:20] <tristan> ploum, this may have problems that I'm not considering at the moment
+[11:41:27] <tristan> I would try it yeah
+[11:41:32] <ploum> thanks a lot