Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 178 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.4783. Min Range: -6.4783

Max Act: 6.4783. Min Act: -0.1700

Data Index: 6061116 (Open Web Text)

Max Activating Token Index: 973

Text #1

Max Range: 6.4783. Min Range: -6.4783

Max Act: 6.1452. Min Act: -0.1700

Data Index: 8672152 (Open Web Text)

Max Activating Token Index: 259

Text #2

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.8717. Min Act: -0.1700

Data Index: 3106364 (Open Web Text)

Max Activating Token Index: 620

Text #3

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.7768. Min Act: -0.1700

Data Index: 456835 (Open Web Text)

Max Activating Token Index: 455

Text #4

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.6549. Min Act: -0.1700

Data Index: 8580178 (Open Web Text)

Max Activating Token Index: 426

Text #5

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.6867. Min Act: -0.1700

Data Index: 2703897 (Open Web Text)

Max Activating Token Index: 493

Text #6

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.6304. Min Act: -0.1700

Data Index: 4784979 (Open Web Text)

Max Activating Token Index: 685

Text #7

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5811. Min Act: -0.1700

Data Index: 4927879 (Open Web Text)

Max Activating Token Index: 782

Text #8

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.6146. Min Act: -0.1700

Data Index: 4527368 (Open Web Text)

Max Activating Token Index: 414

Text #9

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5612. Min Act: -0.1700

Data Index: 2563338 (Open Web Text)

Max Activating Token Index: 747

Text #10

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5570. Min Act: -0.1700

Data Index: 3882718 (Open Web Text)

Max Activating Token Index: 673

Text #11

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5638. Min Act: -0.1700

Data Index: 5737468 (Open Web Text)

Max Activating Token Index: 78

Text #12

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5032. Min Act: -0.1700

Data Index: 5431526 (Open Web Text)

Max Activating Token Index: 853

Text #13

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5056. Min Act: -0.1700

Data Index: 6816613 (Open Web Text)

Max Activating Token Index: 673

Text #14

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5289. Min Act: -0.1700

Data Index: 2213126 (Open Web Text)

Max Activating Token Index: 799

Text #15

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.5083. Min Act: -0.1700

Data Index: 4967955 (Open Web Text)

Max Activating Token Index: 324

Text #16

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.4613. Min Act: -0.1700

Data Index: 6660569 (Open Web Text)

Max Activating Token Index: 882

Text #17

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.4070. Min Act: -0.1700

Data Index: 4142903 (Open Web Text)

Max Activating Token Index: 266

Text #18

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.3626. Min Act: -0.1700

Data Index: 8558711 (Open Web Text)

Max Activating Token Index: 539

Text #19

Max Range: 6.4783. Min Range: -6.4783

Max Act: 5.3736. Min Act: -0.1700

Data Index: 8528955 (Open Web Text)

Max Activating Token Index: 364