Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2889 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.9765. Min Act: -0.0000

Data Index: 1076378 (The Pile)

Max Activating Token Index: 424

Text #1

Max Range: 1.1085. Min Range: -1.1085

Max Act: 1.1085. Min Act: -0.0000

Data Index: 1609242 (The Pile)

Max Activating Token Index: 720

Text #2

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.9192. Min Act: -0.0000

Data Index: 339154 (The Pile)

Max Activating Token Index: 711

Text #3

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8390. Min Act: -0.0000

Data Index: 1046349 (The Pile)

Max Activating Token Index: 101

Text #4

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7926. Min Act: -0.0000

Data Index: 1567201 (The Pile)

Max Activating Token Index: 473

Text #5

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8589. Min Act: -0.0000

Data Index: 1113493 (The Pile)

Max Activating Token Index: 642

Text #6

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8279. Min Act: -0.0000

Data Index: 34136 (The Pile)

Max Activating Token Index: 308

Text #7

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8287. Min Act: -0.0000

Data Index: 1747517 (The Pile)

Max Activating Token Index: 536

Text #8

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7949. Min Act: -0.0000

Data Index: 636255 (The Pile)

Max Activating Token Index: 662

Text #9

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8187. Min Act: -0.0000

Data Index: 756533 (The Pile)

Max Activating Token Index: 444

Text #10

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8166. Min Act: -0.0000

Data Index: 430412 (The Pile)

Max Activating Token Index: 535

Text #11

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7906. Min Act: -0.0000

Data Index: 1682600 (The Pile)

Max Activating Token Index: 249

Text #12

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8252. Min Act: -0.0000

Data Index: 359776 (The Pile)

Max Activating Token Index: 343

Text #13

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.8178. Min Act: -0.0000

Data Index: 639106 (The Pile)

Max Activating Token Index: 392

Text #14

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.6870. Min Act: -0.0000

Data Index: 1334622 (The Pile)

Max Activating Token Index: 806

Text #15

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7233. Min Act: -0.0000

Data Index: 1938281 (The Pile)

Max Activating Token Index: 475

Text #16

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7617. Min Act: -0.0000

Data Index: 1075184 (The Pile)

Max Activating Token Index: 214

Text #17

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7582. Min Act: -0.0000

Data Index: 901859 (The Pile)

Max Activating Token Index: 816

Text #18

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7193. Min Act: -0.0000

Data Index: 631878 (The Pile)

Max Activating Token Index: 51

Text #19

Max Range: 1.1085. Min Range: -1.1085

Max Act: 0.7133. Min Act: -0.0000

Data Index: 43080 (The Pile)

Max Activating Token Index: 520