Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1516 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.2514. Min Act: -0.1700

Data Index: 5915414 (Open Web Text)

Max Activating Token Index: 268

Text #1

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.1613. Min Act: -0.1700

Data Index: 8048583 (Open Web Text)

Max Activating Token Index: 833

Text #2

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.1104. Min Act: -0.1700

Data Index: 3156554 (Open Web Text)

Max Activating Token Index: 1001

Text #3

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0771. Min Act: -0.1700

Data Index: 4857532 (Open Web Text)

Max Activating Token Index: 268

Text #4

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0722. Min Act: -0.1700

Data Index: 5472037 (Open Web Text)

Max Activating Token Index: 231

Text #5

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0359. Min Act: -0.1700

Data Index: 4893383 (Open Web Text)

Max Activating Token Index: 705

Text #6

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0421. Min Act: -0.1700

Data Index: 6861738 (Open Web Text)

Max Activating Token Index: 269

Text #7

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0165. Min Act: -0.1700

Data Index: 4300324 (Open Web Text)

Max Activating Token Index: 857

Text #8

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0126. Min Act: -0.1700

Data Index: 5272633 (Open Web Text)

Max Activating Token Index: 473

Text #9

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0097. Min Act: -0.1700

Data Index: 8771306 (Open Web Text)

Max Activating Token Index: 583

Text #10

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0131. Min Act: -0.1700

Data Index: 6039851 (Open Web Text)

Max Activating Token Index: 102

Text #11

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9943. Min Act: -0.1700

Data Index: 6254298 (Open Web Text)

Max Activating Token Index: 839

Text #12

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9785. Min Act: -0.1700

Data Index: 6684281 (Open Web Text)

Max Activating Token Index: 1023

Text #13

Max Range: 3.2514. Min Range: -3.2514

Max Act: 3.0059. Min Act: -0.1700

Data Index: 8095777 (Open Web Text)

Max Activating Token Index: 410

Text #14

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9863. Min Act: -0.1700

Data Index: 4413638 (Open Web Text)

Max Activating Token Index: 864

Text #15

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9637. Min Act: -0.1700

Data Index: 4115062 (Open Web Text)

Max Activating Token Index: 339

Text #16

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9764. Min Act: -0.1700

Data Index: 213148 (Open Web Text)

Max Activating Token Index: 726

Text #17

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9846. Min Act: -0.1700

Data Index: 2936021 (Open Web Text)

Max Activating Token Index: 17

Text #18

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9886. Min Act: -0.1700

Data Index: 4443533 (Open Web Text)

Max Activating Token Index: 177

Text #19

Max Range: 3.2514. Min Range: -3.2514

Max Act: 2.9707. Min Act: -0.1700

Data Index: 6140276 (Open Web Text)

Max Activating Token Index: 334