Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1175 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.3740. Min Act: -0.1700

Data Index: 1208592 (Open Web Text)

Max Activating Token Index: 1002

Text #1

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.3317. Min Act: -0.1700

Data Index: 5360727 (Open Web Text)

Max Activating Token Index: 930

Text #2

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2985. Min Act: -0.1700

Data Index: 6056051 (Open Web Text)

Max Activating Token Index: 41

Text #3

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2370. Min Act: -0.1700

Data Index: 7810764 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2370. Min Act: -0.1700

Data Index: 3899541 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2357. Min Act: -0.1700

Data Index: 3485139 (Open Web Text)

Max Activating Token Index: 243

Text #6

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2370. Min Act: -0.1700

Data Index: 8098324 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2016. Min Act: -0.1700

Data Index: 2725682 (Open Web Text)

Max Activating Token Index: 209

Text #8

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2354. Min Act: -0.1700

Data Index: 5138798 (Open Web Text)

Max Activating Token Index: 595

Text #9

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.2097. Min Act: -0.1700

Data Index: 6552372 (Open Web Text)

Max Activating Token Index: 11

Text #10

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1985. Min Act: -0.1700

Data Index: 8026693 (Open Web Text)

Max Activating Token Index: 540

Text #11

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1504. Min Act: -0.1700

Data Index: 7510549 (Open Web Text)

Max Activating Token Index: 792

Text #12

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1379. Min Act: -0.1700

Data Index: 4668150 (Open Web Text)

Max Activating Token Index: 382

Text #13

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1149. Min Act: -0.1700

Data Index: 8656193 (Open Web Text)

Max Activating Token Index: 514

Text #14

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1220. Min Act: -0.1700

Data Index: 707484 (Open Web Text)

Max Activating Token Index: 383

Text #15

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.0916. Min Act: -0.1700

Data Index: 5122781 (Open Web Text)

Max Activating Token Index: 72

Text #16

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1201. Min Act: -0.1700

Data Index: 1037332 (Open Web Text)

Max Activating Token Index: 35

Text #17

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.1088. Min Act: -0.1700

Data Index: 7261061 (Open Web Text)

Max Activating Token Index: 9

Text #18

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.0784. Min Act: -0.1700

Data Index: 297409 (Open Web Text)

Max Activating Token Index: 187

Text #19

Max Range: 4.3740. Min Range: -4.3740

Max Act: 4.0971. Min Act: -0.1700

Data Index: 7483196 (Open Web Text)

Max Activating Token Index: 5