Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1159 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4998. Min Act: -0.1700

Data Index: 973407 (Open Web Text)

Max Activating Token Index: 272

Text #1

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.5083. Min Act: -0.1700

Data Index: 5087255 (Open Web Text)

Max Activating Token Index: 508

Text #2

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.5100. Min Act: -0.1700

Data Index: 3461652 (Open Web Text)

Max Activating Token Index: 387

Text #3

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4648. Min Act: -0.1700

Data Index: 8164867 (Open Web Text)

Max Activating Token Index: 866

Text #4

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4709. Min Act: -0.1700

Data Index: 5696710 (Open Web Text)

Max Activating Token Index: 608

Text #5

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4456. Min Act: -0.1700

Data Index: 4563171 (Open Web Text)

Max Activating Token Index: 940

Text #6

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4609. Min Act: -0.1700

Data Index: 1871943 (Open Web Text)

Max Activating Token Index: 479

Text #7

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4607. Min Act: -0.1700

Data Index: 4371761 (Open Web Text)

Max Activating Token Index: 495

Text #8

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4516. Min Act: -0.1700

Data Index: 4016682 (Open Web Text)

Max Activating Token Index: 964

Text #9

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4347. Min Act: -0.1700

Data Index: 6523988 (Open Web Text)

Max Activating Token Index: 548

Text #10

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4149. Min Act: -0.1700

Data Index: 234625 (Open Web Text)

Max Activating Token Index: 414

Text #11

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4286. Min Act: -0.1700

Data Index: 1932449 (Open Web Text)

Max Activating Token Index: 1001

Text #12

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4421. Min Act: -0.1700

Data Index: 433767 (Open Web Text)

Max Activating Token Index: 976

Text #13

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4169. Min Act: -0.1700

Data Index: 6128849 (Open Web Text)

Max Activating Token Index: 472

Text #14

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4199. Min Act: -0.1700

Data Index: 4099248 (Open Web Text)

Max Activating Token Index: 804

Text #15

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4206. Min Act: -0.1700

Data Index: 3884238 (Open Web Text)

Max Activating Token Index: 984

Text #16

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4196. Min Act: -0.1700

Data Index: 3532014 (Open Web Text)

Max Activating Token Index: 888

Text #17

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4137. Min Act: -0.1700

Data Index: 7285857 (Open Web Text)

Max Activating Token Index: 680

Text #18

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4118. Min Act: -0.1700

Data Index: 8244657 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 3.5100. Min Range: -3.5100

Max Act: 3.4152. Min Act: -0.1700

Data Index: 2487724 (Open Web Text)

Max Activating Token Index: 777