Connectomics

!pip install --upgrade vfb_connect
!pip install --upgrade git+git://github.com/schlegelp/navis@master
!pip install --upgrade git+git://github.com/schlegelp/pymaid@master
!pip install --upgrade ipywidgets
!pip install --upgrade jsonpath_rw
!pip install --upgrade seaborn
!pip install --upgrade neuprint-python
!pip install --upgrade flybrains

# Import libs and initialise API objects
from vfb_connect.cross_server_tools import VfbConnect
import pandas as pd
vc = VfbConnect()

import pymaid
import navis

navis.set_pbars(jupyter=False)
pymaid.set_pbars(jupyter=False)

# Connect to the VFB CATMAID server hosting the FAFB data
rm = pymaid.connect_catmaid(server="https://fafb.catmaid.virtualflybrain.org/", api_token=None, max_threads=10)

# Test call to see if connection works 
print(f'Server is running CATMAID version {rm.catmaid_version}')

WARNING: Could not load OpenGL library.
INFO  : Global CATMAID instance set. Caching is ON. (pymaid)
Server is running CATMAID version 2020.02.15-905-g93a969b37

VFB provides a simple methods for exploring connectomics data

get_connected_neurons_by_type allows queries for connections between defined upstream and downstream neuron types/classes. This supports a level of abstraction not available without VFB, where grouping of neurons by higher level classifications is limited to extraction of classification from standardised names. The following examples take advantage of the abstract clasifications on VFB to explore direct synaptic connections.

# Find all GABA-regic inputs onto descending neurons

vc.get_connected_neurons_by_type(upstream_type='GABAergic neuron',
                                 downstream_type='adult descending neuron',
                                 weight=10).sort_values('weight', ascending=False, ignore_index=True)

# Get connections between pacemaker neurons
vc.get_connected_neurons_by_type(upstream_type='pacemaker neuron',
                                 downstream_type='pacemaker neuron',
                                 weight=10).sort_values('weight', ascending=False)

# Get connections between visual projection neurons and descending neurons
vc.get_connected_neurons_by_type(upstream_type='visual projection neuron',
                                 downstream_type='adult descending neuron',
                                 weight=10).sort_values('weight', ascending=False)

Get neurons downstream/downstream of a specific neuron

Starting from an individual neuron, in this case an individual neuron from hemibrain, find all neurons upstream or downstream, restricting to edges >= some specified wieght (synaptic connections) and optionally limiting the output by neuron type/class.

# Starting point - let's take one of the individual descending neurons from
# the previous exploratory queries and look for other inputs

vc.get_neurons_upstream_of('DNp04_R - 1405231475', weight = 20)

Let’s try a similar query from one of the LC4 neurons to see what other descending neuron types are downstream of it.

vc.get_neurons_downstream_of('LC4 - 1249932198', classification="'adult descending neuron'", weight = 20)

Running query: FBbt:00047511
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00047511&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 139

More sophisticated connectomics queries require direct queries of connectomics DB APIs

CATMAID connectivity queries

Broadly speaking, using pymaid to query CATMAID servers lets you fetch connectivity data as either lists of up- and downstream partners or as whole adjacency matrices.

These examples use the VFB FAFB server.

# Using VFB to get neurons by type
DA1 = vc.get_instances("'adult antennal lobe projection neuron DA1'", summary=True)
DA1_tab = pd.DataFrame.from_records(DA1)
len(DA1_tab)
DA1_tab

Running query: FBbt:00048096
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00048096&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 31

# Map neurons to CATMAID Skeleton IDs (skids)

da1_skids = vc.neo_query_wrapper.vfb_id_2_xrefs(DA1_tab['id'], db='catmaid_fafb', reverse_return=True)
da1_skids

{'2863104': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010124e'}],
 '57353': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101202'}],
 '2381753': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010123b'}],
 '1811442': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010121x'}],
 '57323': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101201'}],
 '57311': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101200'}],
 '2345089': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010122p'}],
 '27295': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101199'}],
 '2379517': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010122y'}],
 '2380564': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010122z'}],
 '755022': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101205'}],
 '61221': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101204'}],
 '57381': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_00101203'}],
 '3239781': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010124l'}],
 '4207871': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010126e'}],
 '2334841': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010122m'}],
 '2319457': [{'db': 'catmaid_fafb', 'vfb_id': 'VFB_0010122k'}]}

# Map neurons to CATMAID Skeleton IDs (skids)

da1_skids = vc.neo_query_wrapper.vfb_id_2_xrefs(DA1_tab['id'], db='catmaid_fafb', reverse_return=True)
da1_skids_int = list(map(int, da1_skids))
da1_skids_int

[2863104,
 57353,
 2381753,
 1811442,
 57323,
 57311,
 2345089,
 27295,
 2379517,
 2380564,
 755022,
 61221,
 57381,
 3239781,
 4207871,
 2334841,
 2319457]

Generate a connectivity table for neurons downstream of DA1 neurons

da1 = pymaid.get_neurons(da1_skids_int)
da1_ds = pymaid.get_partners(da1_skids_int,
                             threshold=3,  # anything with >= 3 synapses
                             directions=['outgoing']  # downstream partners only
                            )

# Result is a pandas DataFrame
da1_ds.head()

INFO  : Fetching connectivity table for 17 neurons (pymaid)
INFO  : Done. Found 0 pre-, 270 postsynaptic and 0 gap junction-connected neurons (pymaid)

# Get an adjacency matrix between all Bates, Schlegel et al. neurons
bates = pymaid.find_neurons(annotations='Paper: Bates and Schlegel et al 2020')
adj = pymaid.adjacency_matrix(bates)
adj.head()

INFO  : Found 583 neurons matching the search parameters (pymaid)

import seaborn as sns 

ax = sns.clustermap(adj, vmax=10, cmap='Greys')

/shared-libs/python3.7/py/lib/python3.7/site-packages/seaborn/matrix.py:649: UserWarning:

Clustering large matrix with scipy. Installing `fastcluster` may give better performance.

# Axo-axonic connections between two different types of DA1 PNs
cn = pymaid.get_connectors_between(2863104, 1811442)
cn.head()

import numpy as np 

points = np.vstack(cn.connector_loc)

navis.plot3d([da1.idx[[1811442, 57353]],  # plot the two neurons
              points],  # plot the points of synaptic contacts as scatter 
              scatter_kws=dict(name="synaptic contacts")
              )

#neu_da1 = neu.fetch_skeletons(neu.NeuronCriteria(type='DA1_lPN.*', regex=True))
neu_da1

split = navis.split_axon_dendrite(neu_da1[0])
split

navis.plot3d(split, connectors=True)

WARNING : Skipping empty neuron: DA1_lPN_R (722817260) (navis)

cn = neu.fetch_synapse_connections(source_criteria=neu.NeuronCriteria(type='DA1_lPN.*', regex=True),
                                   target_criteria=neu.NeuronCriteria(type='DA1_vPN.*', regex=True))
cn.head()

cn.groupby(['bodyId_pre', 'bodyId_post', 'roi_pre']).size()

bodyId_pre  bodyId_post  roi_pre
722817260   733316908    AL(R)      26
                         LH(R)      28
754534424   733316908    AL(R)      13
                         LH(R)      31
754538881   733316908    AL(R)      33
                         LH(R)       5
1734350788  733316908    AL(R)      21
                         LH(R)      38
1734350908  733316908    AL(R)      24
                         LH(R)      37
1765040289  733316908    AL(R)      27
                         LH(R)       2
5813039315  733316908    AL(R)      12
                         LH(R)      33
dtype: int64

neuprint connectivity queries

To programmatically interface with neuprint, we will use neuprint-python (link). It requires an API token which you can get via the website and is bound to the Google account that you use to log into neuprint. For this workshop we provide such a token as environment variable but you will need to start using your own token after the workshop is over.

These examples use the hemibrain v1.1 dataset.

neuprint-python

First we have to initialize the connection.

import neuprint as neu
client = neu.Client('https://neuprint.janelia.org',
                    dataset='hemibrain:v1.1',
                    token='eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InZmYndvcmtzaG9wLm5ldXJvZmx5MjAyMEBnbWFpbC5jb20iLCJsZXZlbCI6Im5vYXV0aCIsImltYWdlLXVybCI6Imh0dHBzOi8vbGg2Lmdvb2dsZXVzZXJjb250ZW50LmNvbS8tWXFDN21NRXd3TlEvQUFBQUFBQUFBQUkvQUFBQUFBQUFBQUEvQU1adXVjbU5zaXhXZDRhM0VyTTQ0ODBMa2IzNDdvUlpfUS9zOTYtYy9waG90by5qcGc_c3o9NTA_c3o9NTAiLCJleHAiOjE3OTQwOTE4ODd9.ceg4mrj2o-aOhK0NHNGmBacg8R34PBPoLBwhCo4uOCQ')

Most functions in neuprint-python accept neu.NeuronCriteria which is effectively a filter for body IDs, types, cellbody fibers (~lineages), etc:

help(neu.NeuronCriteria)

Help on class NeuronCriteria in module neuprint.neuroncriteria:

class NeuronCriteria(builtins.object)
 |  NeuronCriteria(matchvar='n', *, bodyId=None, instance=None, type=None, regex=False, cellBodyFiber=None, status=None, cropped=None, min_pre=0, min_post=0, rois=None, inputRois=None, outputRois=None, min_roi_inputs=1, min_roi_outputs=1, label=None, roi_req='all', client=None)
 |  
 |  Specifies which fields to filter by when searching for a Neuron (or Segment).
 |  This class does not send queries itself, but you use it to specify search
 |  criteria for various query functions.
 |  
 |  Note:
 |      For simple queries involving only particular bodyId(s) or type(s)/instance(s),
 |      you can usually just pass the ``bodyId`` or ``type`` to the query function,
 |      without constructing a full ``NeuronCriteria``.
 |  
 |      .. code-block:: python
 |  
 |          from neuprint import fetch_neurons, NeuronCriteria as NC
 |  
 |          # Equivalent
 |          neuron_df, conn_df = fetch_neurons(NC(bodyId=329566174))
 |          neuron_df, conn_df = fetch_neurons(329566174)
 |  
 |          # Equivalent
 |          # (Criteria is satisfied if either type or instance matches.)
 |          neuron_df, conn_df = fetch_neurons(NC(type="OA-VPM3", instance="OA-VPM3"))
 |          neuron_df, conn_df = fetch_neurons("OA-VPM3")
 |  
 |  Methods defined here:
 |  
 |  __eq__(self, value)
 |      Implement comparison between criteria.
 |      Note: 'matchvar' is not considered during the comparison.
 |  
 |  __init__(self, matchvar='n', *, bodyId=None, instance=None, type=None, regex=False, cellBodyFiber=None, status=None, cropped=None, min_pre=0, min_post=0, rois=None, inputRois=None, outputRois=None, min_roi_inputs=1, min_roi_outputs=1, label=None, roi_req='all', client=None)
 |      Except for ``matchvar``, all parameters must be passed as keyword arguments.
 |      
 |      .. note::
 |      
 |          **Options for specifying ROI criteria**
 |      
 |          The ``rois`` argument merely matches neurons that intersect the given ROIs at all
 |          (without distinguishing between inputs and outputs).
 |      
 |          The ``inputRois`` and ``outputRois`` arguments allow you to put requirements
 |          on whether or not neurons have inputs or outputs in the listed ROIs.
 |          It results a more expensive query, but its more powerful.
 |          It also enables you to require a minimum number of connections in the given
 |          ``inputRois`` or ``outputRois`` using the ``min_roi_inputs`` and ``min_roi_outputs``
 |          criteria.
 |      
 |          In either case, use use ``roi_req`` to specify whether a neuron must match just
 |          one (``any``) of the listed ROIs, or ``all`` of them.
 |      
 |      Args:
 |          matchvar (str):
 |              An arbitrary cypher variable name to use when this
 |              ``NeuronCriteria`` is used to construct cypher queries.
 |              To help catch errors (such as accidentally passing a ``type`` or
 |              ``instance`` name in the wrong argument position), we require that
 |              ``matchvar`` begin with a lowercase letter.
 |      
 |          bodyId (int or list of ints):
 |              List of bodyId values.
 |      
 |          instance (str or list of str):
 |              If ``regex=True``, then the instance will be matched as a regular expression.
 |              Otherwise, only exact matches are found. To search for neurons with no instance
 |              at all, use ``instance=[None]``. If both ``type`` and ``instance`` criteria are
 |              supplied, any neuron that matches EITHER criteria will match the overall criteria.
 |      
 |          type (str or list of str):
 |              If ``regex=True``, then the type will be matched as a regular expression.
 |              Otherwise, only exact matches are found. To search for neurons with no type
 |              at all, use ``type=[None]``. If both ``type`` and ``instance`` criteria are
 |              supplied, any neuron that matches EITHER criteria will match the overall criteria.
 |      
 |          regex (bool):
 |              If ``True``, the ``instance`` and ``type`` arguments will be interpreted as
 |              regular expressions, rather than exact match strings.
 |      
 |          cellBodyFiber (str or list of str):
 |              Matches for the neuron ``cellBodyFiber`` field.  To search for neurons
 |              with no CBF at all, use ``cellBodyFiber=[None]``.
 |      
 |          status (str or list of str):
 |              Matches for the neuron ``status`` field.  To search for neurons with no status
 |              at all, use ``status=[None]``.
 |      
 |          cropped (bool):
 |              If given, restrict results to neurons that are cropped or not.
 |      
 |          min_pre (int):
 |              Exclude neurons that don't have at least this many t-bars (outputs) overall,
 |              regardless of how many t-bars exist in any particular ROI.
 |      
 |          min_post (int):
 |              Exclude neurons that don't have at least this many PSDs (inputs) overall,
 |              regardless of how many PSDs exist in any particular ROI.
 |      
 |          rois (str or list of str):
 |              ROIs that merely intersect the neuron, without specifying whether
 |              they're intersected by input or output synapses.
 |              If not provided, will be auto-set from ``inputRois`` and ``outputRois``.
 |      
 |          inputRois (str or list of str):
 |              Only Neurons which have inputs in EVERY one of the given ROIs will be matched.
 |              ``regex`` does not apply to this parameter.
 |      
 |          outputRois (str or list of str):
 |              Only Neurons which have outputs in EVERY one of the given ROIs will be matched.
 |              ``regex`` does not apply to this parameter.
 |      
 |          min_roi_inputs (int):
 |              How many input (post) synapses a neuron must have in each ROI to satisfy the
 |              ``inputRois`` criteria.  Can only be used if you provided ``inputRois``.
 |      
 |          min_roi_outputs (int):
 |              How many output (pre) synapses a neuron must have in each ROI to satisfy the
 |              ``outputRois`` criteria.   Can only be used if you provided ``outputRois``.
 |      
 |          roi_req (Either ``'any'`` or ``'all'``):
 |              Whether a neuron must intersect all of the listed input/output ROIs, or any of the listed input/output ROIs.
 |              When using 'any', each neuron must still match at least one input AND at least one output ROI.
 |      
 |          label (Either ``'Neuron'`` or ``'Segment'``):
 |              Which node label to match with.
 |              (In neuprint, all ``Neuron`` nodes are also ``Segment`` nodes.)
 |              By default, ``'Neuron'`` is used, unless you provided a non-empty ``bodyId`` list.
 |              In that case, ``'Segment'`` is the default. (It's assumed you're really interested
 |              in the bodies you explicitly listed, whether or not they have the ``'Neuron'`` label.)
 |      
 |          client (:py:class:`neuprint.client.Client`):
 |              Used to validate ROI names.
 |              If not provided, the global default ``Client`` will be used.
 |  
 |  __repr__(self)
 |      Return repr(self).
 |  
 |  all_conditions(self, *vars, prefix=0, comments=True)
 |  
 |  basic_conditions(self, prefix=0, comments=True)
 |      Construct a WHERE clause based on the basic conditions
 |      in this criteria (i.e. everything except for the "directed ROI" conditions.)
 |  
 |  basic_exprs(self)
 |      Return the list of expressions that correspond
 |      to the members in this NeuronCriteria object.
 |      They're intended be combined (via 'AND') in
 |      the WHERE clause of a cypher query.
 |  
 |  bodyId_expr(self)
 |  
 |  cbf_expr(self)
 |  
 |  cropped_expr(self)
 |  
 |  directed_rois_condition(self, *vars, prefix=0, comments=True)
 |      Construct the ```WITH...WHERE``` statements that apply the "directed ROI"
 |      conditions specified by this criteria's ``inputRois`` and ``outputRois``
 |      members.
 |      
 |      These conditions are expensive to evaluate, so it's usually a good
 |      idea to position them LAST in your cypher query, once the result set
 |      has already been narrowed down by eariler filters.
 |  
 |  global_vars(self)
 |  
 |  global_with(self, *vars, prefix=0)
 |  
 |  instance_expr(self)
 |  
 |  post_expr(self)
 |  
 |  pre_expr(self)
 |  
 |  rois_expr(self)
 |  
 |  status_expr(self)
 |  
 |  type_expr(self)
 |  
 |  typeinst_expr(self)
 |      Unlike all other fields, type and instance OR'd together.
 |      Either match satisfies the criteria.
 |  
 |  ----------------------------------------------------------------------
 |  Class methods defined here:
 |  
 |  combined_conditions(neuron_conditions, vars=[], prefix=0, comments=True) from builtins.type
 |      Combine the conditions from multiple NeuronCriteria into a single string,
 |      putting the "cheap" conditions first and the "expensive" conditions last.
 |      (That is, basic conditions first and the directed ROI conditions last.)
 |  
 |  combined_global_with(neuron_conditions, vars=[], prefix=0) from builtins.type
 |  
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |  
 |  __dict__
 |      dictionary for instance variables (if defined)
 |  
 |  __weakref__
 |      list of weak references to the object (if defined)
 |  
 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |  
 |  MAX_LITERAL_LENGTH = 3
 |  
 |  __hash__ = None

Fetching neurons

Let’s say we want to find all antennnal lobe projection neurons (PNs). Their type nomenclature adheres to {glomerulus}_{lineage}PN (e.g. DA1_lPN)for uniglomerular PNs and a M_{lineage}PN{tract}{type} (e.g. M_vPNml50 = “multiglomerular ventral lineage PN mediolateral tract type 50) for multiglomerular PNs.

To get them all, we need to use regex patterns (see this cheatsheet):

# Define the filter criteria
nc = neu.NeuronCriteria(type='.*?_.*?PN.*?', regex=True)

# Get general info for these neurons 
pns, roi_info = neu.fetch_neurons(nc)

print(f'{pns.shape[0]} PNs found.')

pns.head()

337 PNs found.

# Check that the regex did not have any accidental by-catch
pns['type'].unique()

array(['M_vPNml53', 'M_lvPNm32', 'M_lvPNm36', 'M_vPNml70', 'M_vPNml89',
       'VP1l+_lvPN', 'M_vPNml69', 'DM1_lPN', 'DM4_vPN', 'M_vPNml79',
       'VP4+_vPN', 'DA4l_adPN', 'M_vPNml87', 'DM4_adPN', 'M_vPNml83',
       'VA5_lPN', 'DA4m_adPN', 'M_lvPNm24', 'M_vPNml85', 'VP1l+VP3_ilPN',
       'M_vPNml77', 'M_vPNml84', 'VC1_lPN', 'M_lvPNm39', 'M_vPNml50',
       'DM2_lPN', 'VC5_lvPN', 'M_vPNml88', 'M_vPNml58', 'VP4_vPN',
       'DP1m_vPN', 'DP1m_adPN', 'DM5_lPN', 'VC5_adPN', 'M_vPNml80',
       'M_lvPNm25', 'VC3m_lvPN', 'VP3+_vPN', 'VP1m+_lvPN', 'DA3_adPN',
       'V_l2PN', 'M_vPNml56', 'VC3l_adPN', 'VM7v_adPN', 'DL5_adPN',
       'VM4_adPN', 'VM2_adPN', 'M_lvPNm40', 'DC4_vPN', 'V_ilPN',
       'M_vPNml74', 'Z_lvPNm1', 'DA1_lPN', 'DP1l_adPN', 'VM4_lvPN',
       'M_vPNml71', 'DP1l_vPN', 'M_lvPNm41', 'M_spPN5t10', 'DA1_vPN',
       'VC4_adPN', 'DM3_adPN', 'M_lvPNm45', 'VL1_vPN', 'M_lvPNm44',
       'M_vPNml78', 'M_vPNml67', 'M_adPNm5', 'M_smPNm1', 'DM6_adPN',
       'DL2d_adPN', 'M_adPNm6', 'M_adPNm8', 'M_lvPNm43', 'Z_vPNml1',
       'M_vPNml59', 'DA2_lPN', 'M_lPNm11A', 'M_vPNml52', 'DL2d_vPN',
       'VL2p_vPN', 'VA1d_adPN', 'M_lPNm11B', 'M_lvPNm48', 'M_lPNm11C',
       'M_lvPNm42', 'VA1v_vPN', 'M_vPNml68', 'M_vPNml55', 'M_vPNml62',
       'VL2a_vPN', 'M_vPNml60', 'M_vPNml65', 'VM5d_adPN', 'M_l2PNm16',
       'M_vPNml61', 'M_vPNml57', 'M_vPNml64', 'M_lv2PN9t49',
       'VP2+VC5_l2PN', 'M_spPN4t9', 'M_vPNml66', 'M_vPNml75', 'M_vPNml63',
       'M_vPNml72', 'M_lvPNm38', 'D_adPN', 'M_vPNml76', 'M_vPNml54',
       'DM3_vPN', 'M_vPNml86', 'DL3_lPN', 'VA4_lPN', 'VP1d_il2PN',
       'DC1_adPN', 'M_l2PN3t18', 'M_lvPNm35', 'DL4_adPN', 'M_lvPNm28',
       'M_lvPNm27', 'M_ilPNm90', 'M_l2PNl20', 'M_lvPNm29', 'VA7l_adPN',
       'M_lPNm13', 'M_l2PNl21', 'DL1_adPN', 'M_imPNl92', 'M_vPNml73',
       'M_ilPN8t91', 'M_l2PNm14', 'VP1d+VP4_l2PN1', 'M_lvPNm26',
       'DL2v_adPN', 'VP3+VP1l_ivPN', 'M_lvPNm33', 'VA1v_adPN',
       'VP3+_l2PN', 'M_l2PN10t19', 'VP4+VL1_l2PN', 'M_l2PNl22',
       'M_l2PNm15', 'M_lPNm11D', 'MZ_lv2PN', 'DC2_adPN', 'M_lvPNm46',
       'VC2_lPN', 'VM1_lPN', 'VM3_adPN', 'VM7d_adPN', 'M_lvPNm47',
       'M_lPNm12', 'DC3_adPN', 'VP2+_adPN', 'VP1m+VP2_lvPN2',
       'VP1m+VP2_lvPN1', 'VA6_adPN', 'VA7m_lPN', 'M_adPNm7', 'M_adPNm4',
       'VA1d_vPN', 'VA3_adPN', 'VL1_ilPN', 'M_l2PNl23', 'M_lvPNm31',
       'VP1m+VP5_ilPN', 'VL2p_adPN', 'MZ_lvPN', 'VP2_adPN', 'VA2_adPN',
       'VM5v_adPN', 'VP5+VP2_l2PN', 'VP5+VP3_l2PN', 'VP5+_l2PN',
       'M_vPNml51', 'M_smPN6t2', 'M_lvPNm37', 'M_vPNml82', 'M_adPNm3',
       'VP1m_l2PN', 'DC4_adPN', 'VP5+Z_adPN', 'VL2a_adPN', 'VP2_l2PN',
       'M_lvPNm34', 'VP2+Z_lvPN', 'M_lvPNm30', 'M_l2PNm17', 'M_vPNml81',
       'VP1d+VP4_l2PN2'], dtype=object)

Find neuron types with VFB instead (this allows curated neuron classes to be used):

#This will get all ALPNs from ALL datasets on VFB
ALPNs = vc.get_instances("'adult antennal lobe projection neuron'", summary=True)
ALPNs = pd.DataFrame.from_records(ALPNs)
#Select only rows from Hemibrain1.1 dataset
ALPNs=ALPNs[ALPNs['data_source'].str.match('neuprint_JRC_Hemibrain_1point1')]
ALPNs

Running query: FBbt:00067123
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00067123&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 1469

Fetching synaptic partners

Looks good! Next: What’s downstream of those PNs?

ds = neu.fetch_simple_connections(upstream_criteria=list(map(int, ALPNs['accession'])))
#using hemibrain regex
#ds = neu.fetch_simple_connections(upstream_criteria=neu.NeuronCriteria(bodyId=pns.bodyId.values))
ds

Each row is now a connections from a single up- to a single downstream neuron. The “weight” is the number of synapses between the pre- and the postsynaptic neuron. Let’s simplify by grouping by type:

by_type = ds.groupby(['type_pre', 'type_post'], as_index=False).weight.sum()
by_type.sort_values('weight', ascending=False, inplace=True)
by_type.reset_index(drop=True, inplace=True)
by_type

The strongest connections are between PNs and Kenyon Cells (KCs). That’s not much of a surprise since there are thousands of KCs. For the sake of the argument let’s say we want to know where these connections occur:

#First find KCs in VFB
KCs = pd.DataFrame.from_records(vc.get_instances("'adult Kenyon cell'", summary=True))
#Select only rows from Hemibrain1.1 dataset
KCs=KCs[KCs['data_source'].str.match('neuprint_JRC_Hemibrain_1point1')]
KCs

Running query: FBbt:00049825
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00049825&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 5107

adj, roi_info2 = neu.fetch_adjacencies(sources=list(map(int, ALPNs['accession'])), targets=list(map(int, KCs['accession'])))
roi_info2.head()  

  0%|          | 0/2 [00:00<?, ?it/s]

# Group by region of interest (ROI)
by_roi = roi_info2.groupby('roi').weight.sum()
by_roi.head()

roi
CA(R)         181274
NotPrimary      2764
PLP(R)            11
SCL(R)           509
SLP(R)          2026
Name: weight, dtype: int64

ax = by_roi.plot.bar()
ax.set_xlabel('')
ax.set_ylabel('PN to KC synapses')

Text(0, 0.5, 'PN to KC synapses')

Querying paths

Let’s say we want to find out how to go from a PN (second order olfactory neurons) all the way to a descending neuron (presumably leading to motor neurons in the VNC).

#First find DNs in VFB
DNs = pd.DataFrame.from_records(vc.get_instances("'adult descending neuron'", summary=True))
#Select only rows from Hemibrain1.1 dataset
DNs=DNs[DNs['data_source'].str.match('neuprint_JRC_Hemibrain_1point1')]
DNs

Running query: FBbt:00047511
Query URL: http://owl.virtualflybrain.org/kbs/vfb/instances?object=FBbt%3A00047511&prefixes=%7B%22FBbt%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FFBbt_%22%2C+%22RO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_%22%2C+%22BFO%22%3A+%22http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FBFO_%22%7D&direct=False
Query results: 139

# First fetch the DNs
dns, _ = neu.fetch_neurons(list(map(int, DNs['accession'])))
#with regex
#dns, _ = neu.fetch_neurons(neu.NeuronCriteria(type='(.*DN[^1]{0,}.*|Giant Fiber)', regex=True))
dns

Neuprint lets you query paths from a single source to a single target. For multi-source or -target queries, your best bet is to download the entire graph and run the queries locally using networkx or igraph.

# Find all paths from A PN to A DNs 
paths = neu.fetch_shortest_paths(upstream_bodyId=list(map(int, ALPNs['accession']))[0],
                                 downstream_bodyId=list(map(int, DNs['accession']))[0],
                                 min_weight=10)
paths        

So it looks like there are three separate 7-hop paths to go from M_vPNml53 to DN1a. Let’s visualize the neurons involved!

Fetching meshes & skeletons

You can fetch skeletons as SWCs directly via neuprint-python. For visualization however it’s easiest to load neuron morphologies via navis. For that navis wraps neuprint-python and adds some convenience functions (see also the tutorial):

# Import the wrapped neuprint-python 
# -> this exposes ALL base functions plus a couple navis-specific extras
import navis
import navis.interfaces.neuprint as neu 

#client = neu.Client('https://neuprint.janelia.org', dataset='hemibrain:v1.1')

# Fetch neurons in the first path
nl = neu.fetch_skeletons(paths.loc[(paths.path == 0), 'bodyId'])
nl

# Let's also get some ROI meshes
al = neu.fetch_roi('AL(R)')
lh = neu.fetch_roi('LH(R)')
ca = neu.fetch_roi('CA(R)')

# Plot
navis.plot3d([nl, lh, al, ca], width=1100)

Plotting graphs

There are various ways of plotting static graphs. In theory Jupyter notebooks lend themselves to interactive graphs too but unfortunately DeepNote does not yet support the required libraries (e.g. ipywidgets). That being said: if you want to run this locally or on Google colab, check out ipycytoscape.

There are numerous options to do this but we will use networkx to plot a static graph:

import networkx as nx 
import numpy as np

# Initialize the graph
G = nx.DiGraph()

# Generate edges from the paths
edges = []
for p in paths.path.unique():
    this_path = paths.loc[(paths.path == p)]
    this_edges = list(zip(this_path.values[:-1], this_path.values[1:]))

    for i in range(this_path.shape[0] - 1):
        edges.append([this_path.bodyId.values[i], this_path.bodyId.values[i + 1], this_path.weight.values[i + 1]])

# Add the edges 
G.add_weighted_edges_from(edges)

# Add some names to the nodes 
nx.set_node_attributes(G, paths.set_index('bodyId')['type'].to_dict(), name='name')

import matplotlib.pyplot as plt 

# Draw using a simple force-directed layout
pos = nx.kamada_kawai_layout(G)

# We could draw everything in one step but this way we have more control over the plot
fig, ax = plt.subplots(figsize=(10, 10))

# Draw nodes
nx.draw_networkx_nodes(G, pos=pos, ax=ax)

# Draw edges
weights = np.array([e[2]['weight'] for e in G.edges(data=True)])
nx.draw_networkx_edges(G, pos=pos, width=(weights / 12).tolist())

# Add node labels 
nx.draw_networkx_labels(G, pos=pos, labels=dict(G.nodes('name')), font_size=14)

# Turn axes of
ax.set_axis_off()

In general, I recommend exporting your graph to e.g. graphml and importing it into e.g. cytoscape if you want to explore an interactive network graph.

nx.write_gml(G, "my_graph.gml")