Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 167 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.7567. Min Act: -0.1700

Data Index: 7123761 (Open Web Text)

Max Activating Token Index: 27

Text #1

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.6424. Min Act: -0.1700

Data Index: 6005159 (Open Web Text)

Max Activating Token Index: 79

Text #2

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.6258. Min Act: -0.1700

Data Index: 6681605 (Open Web Text)

Max Activating Token Index: 147

Text #3

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5895. Min Act: -0.1700

Data Index: 1089355 (Open Web Text)

Max Activating Token Index: 761

Text #4

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.6045. Min Act: -0.1700

Data Index: 3156488 (Open Web Text)

Max Activating Token Index: 22

Text #5

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5978. Min Act: -0.1700

Data Index: 2922107 (Open Web Text)

Max Activating Token Index: 130

Text #6

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5950. Min Act: -0.1700

Data Index: 4256608 (Open Web Text)

Max Activating Token Index: 355

Text #7

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5873. Min Act: -0.1700

Data Index: 726425 (Open Web Text)

Max Activating Token Index: 11

Text #8

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5814. Min Act: -0.1700

Data Index: 518049 (Open Web Text)

Max Activating Token Index: 846

Text #9

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5851. Min Act: -0.1700

Data Index: 2075685 (Open Web Text)

Max Activating Token Index: 77

Text #10

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5762. Min Act: -0.1700

Data Index: 530790 (Open Web Text)

Max Activating Token Index: 22

Text #11

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5793. Min Act: -0.1700

Data Index: 1462560 (Open Web Text)

Max Activating Token Index: 27

Text #12

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5688. Min Act: -0.1700

Data Index: 1605597 (Open Web Text)

Max Activating Token Index: 31

Text #13

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5741. Min Act: -0.1695

Data Index: 4493930 (Open Web Text)

Max Activating Token Index: 159

Text #14

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5771. Min Act: -0.1700

Data Index: 2503038 (Open Web Text)

Max Activating Token Index: 11

Text #15

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5676. Min Act: -0.1700

Data Index: 6643003 (Open Web Text)

Max Activating Token Index: 199

Text #16

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5609. Min Act: -0.1700

Data Index: 6638692 (Open Web Text)

Max Activating Token Index: 838

Text #17

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5676. Min Act: -0.1700

Data Index: 973850 (Open Web Text)

Max Activating Token Index: 52

Text #18

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5492. Min Act: -0.1700

Data Index: 5788214 (Open Web Text)

Max Activating Token Index: 20

Text #19

Max Range: 1.7567. Min Range: -1.7567

Max Act: 1.5472. Min Act: -0.1700

Data Index: 6013983 (Open Web Text)

Max Activating Token Index: 873