Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 453 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.6146. Min Act: -0.1700

Data Index: 1392822 (Open Web Text)

Max Activating Token Index: 775

Text #1

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.5422. Min Act: -0.1700

Data Index: 6637124 (Open Web Text)

Max Activating Token Index: 1018

Text #2

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.5016. Min Act: -0.1700

Data Index: 4422295 (Open Web Text)

Max Activating Token Index: 976

Text #3

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4866. Min Act: -0.1700

Data Index: 4370995 (Open Web Text)

Max Activating Token Index: 860

Text #4

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4724. Min Act: -0.1700

Data Index: 7393643 (Open Web Text)

Max Activating Token Index: 939

Text #5

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4833. Min Act: -0.1700

Data Index: 6606702 (Open Web Text)

Max Activating Token Index: 680

Text #6

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4774. Min Act: -0.1700

Data Index: 7369892 (Open Web Text)

Max Activating Token Index: 57

Text #7

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4542. Min Act: -0.1700

Data Index: 5557334 (Open Web Text)

Max Activating Token Index: 976

Text #8

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4825. Min Act: -0.1700

Data Index: 3895858 (Open Web Text)

Max Activating Token Index: 838

Text #9

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4607. Min Act: -0.1700

Data Index: 260474 (Open Web Text)

Max Activating Token Index: 881

Text #10

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4667. Min Act: -0.1700

Data Index: 7801302 (Open Web Text)

Max Activating Token Index: 990

Text #11

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4618. Min Act: -0.1700

Data Index: 2022551 (Open Web Text)

Max Activating Token Index: 218

Text #12

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4720. Min Act: -0.1700

Data Index: 28652 (Open Web Text)

Max Activating Token Index: 153

Text #13

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4622. Min Act: -0.1700

Data Index: 3880021 (Open Web Text)

Max Activating Token Index: 664

Text #14

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4441. Min Act: -0.1700

Data Index: 6807103 (Open Web Text)

Max Activating Token Index: 339

Text #15

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4540. Min Act: -0.1700

Data Index: 3217681 (Open Web Text)

Max Activating Token Index: 980

Text #16

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4565. Min Act: -0.1700

Data Index: 7512163 (Open Web Text)

Max Activating Token Index: 360

Text #17

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4486. Min Act: -0.1700

Data Index: 1673687 (Open Web Text)

Max Activating Token Index: 933

Text #18

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4552. Min Act: -0.1700

Data Index: 4199377 (Open Web Text)

Max Activating Token Index: 59

Text #19

Max Range: 1.6146. Min Range: -1.6146

Max Act: 1.4673. Min Act: -0.1700

Data Index: 207734 (Open Web Text)

Max Activating Token Index: 918