Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1961 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.6726. Min Act: -0.1700

Data Index: 2620388 (Open Web Text)

Max Activating Token Index: 115

Text #1

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.5237. Min Act: -0.1700

Data Index: 814589 (Open Web Text)

Max Activating Token Index: 722

Text #2

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4710. Min Act: -0.1700

Data Index: 8302810 (Open Web Text)

Max Activating Token Index: 30

Text #3

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4808. Min Act: -0.1700

Data Index: 7736542 (Open Web Text)

Max Activating Token Index: 476

Text #4

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4715. Min Act: -0.1700

Data Index: 7348239 (Open Web Text)

Max Activating Token Index: 103

Text #5

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4641. Min Act: -0.1700

Data Index: 7692237 (Open Web Text)

Max Activating Token Index: 89

Text #6

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4404. Min Act: -0.1700

Data Index: 3260924 (Open Web Text)

Max Activating Token Index: 450

Text #7

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4541. Min Act: -0.1700

Data Index: 357171 (Open Web Text)

Max Activating Token Index: 953

Text #8

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4544. Min Act: -0.1700

Data Index: 7804194 (Open Web Text)

Max Activating Token Index: 142

Text #9

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4238. Min Act: -0.1700

Data Index: 2244424 (Open Web Text)

Max Activating Token Index: 61

Text #10

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4167. Min Act: -0.1700

Data Index: 1783256 (Open Web Text)

Max Activating Token Index: 69

Text #11

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4226. Min Act: -0.1700

Data Index: 357170 (Open Web Text)

Max Activating Token Index: 483

Text #12

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4051. Min Act: -0.1700

Data Index: 189922 (Open Web Text)

Max Activating Token Index: 37

Text #13

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3973. Min Act: -0.1700

Data Index: 6370576 (Open Web Text)

Max Activating Token Index: 591

Text #14

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.4029. Min Act: -0.1696

Data Index: 258694 (Open Web Text)

Max Activating Token Index: 281

Text #15

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3654. Min Act: -0.1700

Data Index: 4188428 (Open Web Text)

Max Activating Token Index: 113

Text #16

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3828. Min Act: -0.1697

Data Index: 1792389 (Open Web Text)

Max Activating Token Index: 683

Text #17

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3773. Min Act: -0.1700

Data Index: 401907 (Open Web Text)

Max Activating Token Index: 444

Text #18

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3504. Min Act: -0.1700

Data Index: 357186 (Open Web Text)

Max Activating Token Index: 745

Text #19

Max Range: 3.6726. Min Range: -3.6726

Max Act: 3.3579. Min Act: -0.1700

Data Index: 2696162 (Open Web Text)

Max Activating Token Index: 336