Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1893 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.6728. Min Act: -0.1700

Data Index: 2247768 (Open Web Text)

Max Activating Token Index: 88

Text #1

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.6958. Min Act: -0.1700

Data Index: 386918 (Open Web Text)

Max Activating Token Index: 483

Text #2

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.6571. Min Act: -0.1700

Data Index: 8398813 (Open Web Text)

Max Activating Token Index: 384

Text #3

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.6458. Min Act: -0.1700

Data Index: 7362096 (Open Web Text)

Max Activating Token Index: 229

Text #4

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.6194. Min Act: -0.1700

Data Index: 2954489 (Open Web Text)

Max Activating Token Index: 1012

Text #5

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5814. Min Act: -0.1700

Data Index: 2866091 (Open Web Text)

Max Activating Token Index: 49

Text #6

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5496. Min Act: -0.1700

Data Index: 8149388 (Open Web Text)

Max Activating Token Index: 515

Text #7

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5288. Min Act: -0.1700

Data Index: 3358482 (Open Web Text)

Max Activating Token Index: 65

Text #8

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5363. Min Act: -0.1700

Data Index: 2846974 (Open Web Text)

Max Activating Token Index: 59

Text #9

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5173. Min Act: -0.1700

Data Index: 7299311 (Open Web Text)

Max Activating Token Index: 95

Text #10

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5160. Min Act: -0.1700

Data Index: 6633455 (Open Web Text)

Max Activating Token Index: 83

Text #11

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5190. Min Act: -0.1700

Data Index: 3783233 (Open Web Text)

Max Activating Token Index: 950

Text #12

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5258. Min Act: -0.1700

Data Index: 5298730 (Open Web Text)

Max Activating Token Index: 52

Text #13

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5179. Min Act: -0.1700

Data Index: 4595325 (Open Web Text)

Max Activating Token Index: 90

Text #14

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.4902. Min Act: -0.1700

Data Index: 4000034 (Open Web Text)

Max Activating Token Index: 879

Text #15

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5401. Min Act: -0.1700

Data Index: 5256349 (Open Web Text)

Max Activating Token Index: 519

Text #16

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5034. Min Act: -0.1700

Data Index: 3589025 (Open Web Text)

Max Activating Token Index: 362

Text #17

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.4954. Min Act: -0.1700

Data Index: 802776 (Open Web Text)

Max Activating Token Index: 502

Text #18

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.5075. Min Act: -0.1700

Data Index: 4491058 (Open Web Text)

Max Activating Token Index: 96

Text #19

Max Range: 5.6958. Min Range: -5.6958

Max Act: 5.4675. Min Act: -0.1700

Data Index: 6354542 (Open Web Text)

Max Activating Token Index: 740