tensorflow - distributed data reader across multiple machines -
is there distributed data reader tensorflow? if not, suggested way handling big datasets on multiple machines?
the distributed inception example here pre-segments data on multiple machines , each workers grabs subset of data available subsets. supported method?
also seems of data readers described in here thread safe couldn't find distributed solution.
we don't have general solution distributed data reading in tensorflow, , it's bit of hard problem since there many different possible requirements around latency, size, , sharding. i'd interested in proposals or patches though!
Comments
Post a Comment